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PILOT  STUDY  TO  EXAMINE  TRAINING  ELIGIBILITY  STANDARDS 


EXECUTIVE  SUMMARY 


Research  Requirement: 

Training  enlistment  standards  govern  classification  of  new  recruits  into  job  training  -  i.e., 
Advanced  Individual  Training  (AIT).  The  cognitive  standards  subset  is  largely  the  purview  of 
school  proponents  in  coordination  with  Army  G- 1 .  These  standards  establish  qualifying  cutoff 
score  levels  on  Area  Aptitude  (AA)  composites,  derived  from  the  Armed  Services  Vocational 
Aptitude  Battery  (ASVAB).  The  purpose  of  this  study  is  to  develop  methodologies  which  can  be 
utilized  to  assist  school  proponents  in  assessing  the  appropriateness  of  their  AA  cutoff  score 
levels. 

Procedure: 

Regression  models  of  pass  /  fail  training  outcomes  over  the  2001  -  2004  period  utilizing  Army 
Training  Requirements  and  Resources  System  (ATRRS)  data  are  specified  and  estimated  for  a 
total  of  80  MOS  with  sufficient  (failure)  cases.  Training  outcome  is  specified  as  a  function  of 
AA  governing  composite,  Soldier  demographic,  and  component  membership  variables.  The 
estimated  models  are  then  applied  to  the  larger  Army  enlisted  contract  population  to  examine  the 
policy  tradeoffs  between  training  eligibility  and  AIT  completion.  For  select  MOS,  the  policy 
analyses  are  examined  more  closely  using  risk  analysis  simulation  methods. 

Findings: 

Predictive  models  were  successfully  developed  and  applied  to  elucidate  the  tradeoffs  brought 
about  through  lowering  /  raising  minimum  enlistment  training  standards.  Based  on  the  relatively 
broad  sample  of  MOS  examined,  the  usefulness  of  the  estimated  models  for  conducting  policy 
analysis  suggests  three  major  groupings  of  MOS.  In  the  first  group  are  MOS  where  cutoff  score 
levels  are  relatively  low  and  passing  rates  are  uniformly  high.  The  policy  analyses  for  many  of 
these  MOS  suggest  that  cutoff  score  levels  could  be  lowered  further  without  paying  a  noticeable 
price  in  reduced  passing  rates.  In  the  second  group  are  MOS  where  cutoff  score  levels  are 
relatively  high  and  passing  rates  are  lower,  estimated  composite  effects  are  relatively  high,  and 
predicted  passing  rates  are  more  spread  out  in  the  vicinity  of  the  current  cutoff  level.  For  these 
MOS  the  estimated  models  appear  useful  in  revealing  tradeoffs  as  cutoff  score  levels  are 
changed.  In  a  third  group  of  MOS,  the  explanatory  power  of  the  model  is  low  or  non-existent. 

In  most  but  not  all  of  these  the  poor  fit  is  presumed  due  to  too  few  failure  case  observations,  but 
a  definitive  conclusion  is  not  possible. 

Utilization  and  Dissemination  of  Findings: 

Study  findings  establish  the  feasibility  of  examining  the  appropriateness  of  cutoff  score  levels 
with  readily  available  Army  data,  and  should  encourage  school  proponents  to  examine  this  issue 
periodically.  The  findings  were  also  used  to  inform  a  G-l  decision  not  to  support  a  recent 
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proposal  for  across-the-board  reductions  in  cutoff  score  levels.  Finally,  the  findings  constitute 
additional  support  in  ongoing  ASVAB  validation  efforts. 
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Pilot  Study  To  Examine  Training  Eligibility  Standards 


Introduction 

Army  personnel  managers  frequently  need  to  make  tradeoffs  between  Soldiers’  numbers, 
quality,  training  difficulty,  and  other  factors  when  making  personnel  management  and 
training  decisions.  Often,  data  pertaining  to  such  tradeoffs  exist  but  are  not  integrated 
into  the  decision  process  because  they  are  not  readily  accessible. 

Prospective  Soldiers  are  recruited  and  classified  using  quality  marks  that  are 
defined,  in  part,  by  minimum  enlistment  standards.  When  quality  standards  are  set  too 
high  in  an  MOS  it  is  more  difficult  to  make  its  recruiting  target,  and  when  quality 
standards  are  set  too  low  training  performance  suffers  and  attrition  becomes  a  problem. 
The  importance  of  appropriate  quality  marks  is  especially  important  when  recruiting  is 
difficult.  This  is  evidenced  by  the  number  of  recent  calls  to  ARI  from  school  proponents 
asking  for  analysis  of  the  impact  of  lowering  standards. 

Two  years  ago  ARI  completed  a  small  research  effort  that  dealt  with  training 
eligibility  standards  in  four  MOS  -  see  Examining  Training  Eligibility  Standards:  Four 
Case  Studies,  ARI  Study  Note  2005-01,  October  2004.  The  research  focused  on  the 
tradeoff  between  training  eligibility  and  training  perfonnance,  and  grew  out  of  a  project 
sponsored  by  Human  Resources  Command.  This  research  utilized  detailed  training  data 
provided  by  the  Army  Training  Support  Center  (ATSC)  Automated  Instructional 
Management  System.  From  what  we  can  tell,  this  kind  of  training  performance  data  is 
not  available  for  very  many  MOS. 

The  aim  of  this  study  is  to  examine  training  enlistment  standards  utilizing  Army 
Training  Requirements  and  Resources  System  (ATRRS)  data  which  records  individual- 
level  training  events  for  all  MOS,  but  is  limited  to  pass  /  fail  outcomes.  Specifically,  we 
seek  to  investigate  the  tradeoffs  between  training  eligibility  and  training  completion  that 
are  brought  about  by  lowering  /  raising  minimum  enlistment  training  standards  on  the 
“governing”  composite,  and  to  develop  methodologies  which  can  be  used  to  assist  school 
proponents  in  assessing  the  appropriateness  of  their  AA  cut  scores.  For  the  initial  effort 
in  this  pilot,  the  50  MOS  investigated  belonged  to  a  handful  of  school  proponents  who 
expressed  an  interest  in  the  objectives  of  this  study,  plus  a  few  additional  ones 
recommended  by  Army  Accessions  Command.  Subsequently,  we  included  an  additional 
30  MOS  that  promised  sufficient  numbers  of  observations  and  representation  of  most  of 
the  Career  Management  Fields  (CMF). 

While  it  is  reasonable  to  consider  the  impact  of  cut  scores  on  training  eligibility,  it 
should  be  understood  that  that  is  not  the  only  purpose  of  cut  scores.  Cut  scores  are  also 
designed  to  identify  Soldiers  who  will  be  successful  in  their  jobs  after  training.  If 
training  requirements  are  less  demanding  than  job  requirements,  then  cut  scores  which 
are  based  solely  on  training  success  may  not  succeed  in  discriminating  between  those 
who  will  be  successful  on  the  job  from  those  who  will  not. 
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Approach 


Perhaps  the  most  important  caveat  to  this  study  and  the  approach  taken  is  that  we 
restricted  the  analysis  to  so-called  academic  cases.  We  take  at  face  value  the  infonnation 
contained  in  the  data  base,  and  have  no  way  to  verify  the  categorization  of  outcomes  into 
those  due  to  academic  versus  non-academic  reasons.  In  so  doing,  we  assume  that  non¬ 
completion  for  non-academic  reasons  is  not  related  to  cognitive  criteria. 1 

With  the  data  readily  available,  we  are  able  to  examine  the  relationship  between 
Advanced  Individual  Training  (AIT)  outcome  and  Soldier  score  on  the  governing 
Aptitude  Area  (AA)  composite  for  the  particular  MOS,  controlling  for  Soldier  gender  and 
education  level,  and  Soldier  component  membership.  As  mentioned,  the  outcome 
measure  was  restricted  to  pass  /  fail,  and  we  excluded  non-academic  failure  cases.  In 
addition,  we  focused  on  the  first  training  attempt  made  by  the  Soldier. 

We  specified  and  estimated  a  logistic  regression  model  to  explain  the  Soldier’s 
pass  /  fail  training  outcome  as  a  function  of  Aptitude  Area  (AA)  scores  and  other 
variables  just  mentioned.  The  relationship  between  training  outcome  probability  P(Z), 
AA  score,  and  other  variables  can  be  stated  as 

Prob  (outcome)  =  P(Z)  =  ez  /  (l+ez), 

where  Z  =  [3o  +  (fiX|  +  P2X2  +  P3X3  +  . . .;  X  are  the  independent  explanatory  variables;  P 
are  the  estimated  parameters;  and  “e”  is  base  of  the  natural  logarithms.  The  logistic 
regression  curve  is  S-shaped.  The  relationship  between  the  independent  variables  and  the 
probability  of  passing  is  nonlinear.  The  probability  estimates  will  always  be  between 
zero  and  one,  regardless  of  the  value  of  Z.  Note  that  the  increase  in  the  odds  of  passing  is 
given  by  the  factor  ek^  for  a  k-point  increase  in  the  governing  composite  score. 

For  each  MOS,  we  followed  these  steps:  (a)  estimate  a  binary  logistic  regression 
equation;  (b)  examine  the  explanatory  and  predictive  power  of  the  estimated  model  using 
goodness-of-fit  statistics;  (c)  analyze  the  impact  of  changing  the  cutoff  score  upon 
eligibility  and  the  probability  of  successfully  completing  the  training.  Policy 
recommendations  per  se  are  not  made. 

The  policy  analysis  is  conducted  with  respect  to  the  larger  Anny  enlisted  contract 
population.2  Using  the  contract  population,  for  a  given  cutoff  score  we  determine  the 
corresponding  size  of  the  eligible  pool,  and  express  it  as  a  percentage  eligible.  Next,  we 
calculate  the  mean  governing  composite  score  for  the  eligible  pool,  and  then  use  the 


1  We  conducted  additional  empirical  analyses  for  55D,  74B,  91W,  and  96B,  and  found  results  that  were 
consistent  with  the  assumption.  These  analyses  are  offered  as  illustrative  examples;  more  definitive 
support  would  entail  analyses  of  a  larger  number  of  MOS. 

2  Army  contract  data  are  extracted  from  USAREC’s  Enlisted  Accessions  File.  For  most  of  the  analyses, 
we  employed  extracts  for  the  Apr  2002  through  Aug  2003  period;  for  the  remaining  analyses  we  employed 
extracts  for  the  Jan  1992  through  Aug  2003  period.  The  differences  in  derived  statistics  over  these  two 
periods  were  negligible. 
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estimated  regression  model  to  calculate  the  predicted  passing  rate  for  the  average  Soldier 
in  that  pool,  i.e.  for  Soldiers  scoring  at  the  average  or  mean  of  the  eligible  pool.  The 
impact  of  a  cutoff  score  change  is  provided  by  comparing  the  baseline  percentage  eligible 
and  average  passing  rate  with  that  under  a  policy  case.  The  impact  of  the  policy  case  can 
also  be  evaluated  at  other  points  besides  the  mean  of  the  eligible  group  by  identifying  and 
calculating  what  those  points  of  interest  are  and  looking  them  up  on  a  graph  which  shows 
how  the  probability  of  passing  varies  with  the  governing  composite  score.  Graphs  are 
provided  in  all  of  the  analyses. 

An  example  taken  from  the  policy  analysis  of  MOS  96B  (Intelligence  Analyst) 
will  help  to  illustrate  the  steps  involved  (see  Table  1).  Skilled  Technical  (ST)  is  the 
governing  composite  for  the  MOS,  and  the  current  cutoff  score  (circa  2003)  is  102. 

Given  that  cutoff  level,  65%  of  Soldiers  would  be  eligible  for  this  training  assignment, 
and  the  mean  ST  score  of  this  pool  is  1 14.  At  this  score,  the  predicted  passing  rate  is 
72.2%  for  high  school  male  graduates.  Should  the  proponent  consider  a  5  point  reduction 
in  the  cutoff  level  (from  102  to  97),  the  eligible  pool  would  increase  by  13  percentage 
points,  the  average  ST  score  would  fall  to  112,  and  the  corresponding  predicted  passing 
rate  would  fall  to  67%  for  the  average  high  school  male  Soldier  in  the  newly  enlarged 
pool.  Thus,  the  tradeoff  facing  the  proponent  can  be  summarized  as  a  13  percentage 
point  gain  in  the  size  of  the  eligible  pool  at  a  cost  of  5  percentage  points  in  the  probability 
of  passing  for  the  average  Soldier. 

Table  1 .  Probability  that  a  Regular  Anny  Soldier  (from  the  larger  Army  contract 


population)  will  pass  t 

le  course  based  on  the  estimated  regression  model. 

ST  Cutoff 
=  92 

ST  Cutoff 
=  97 

ST  Cutoff 
=  102 

ST  Cutoff 
=  107 

ST  Cutoff 
=  112 

Percent  Eligible 
(Regular  Army) 

89.0% 

78.2% 

64.9% 

49.6%o 

34.7%, 

Mean 

109.45 

111.59 

114.16 

117.18 

120.51 

Passing  Rates: 

High  School  Male 

College  Male 

79.5% 

83.6% 

87.4% 

High  School  Female 

87.1% 

College  Female 

78.5% 

81.7% 

84.9% 

88.1% 

90.9% 

For  select  MOS,  we  also  examined  the  predictive  power  of  the  regression  model 
and  its  application  to  policy  analysis  using  risk  analysis.  Risk  analysis  uses  simulation 
methods  to  take  into  account  all  the  sources  of  uncertainty  found  in  the  regression  model 
and  to  consider  the  impact  of  policy  changes  not  just  at  the  mean  composite  score  (i.e., 
for  the  average  Soldier)  but  over  the  entire  range  of  Soldier  characteristics.  The  risk 
analysis  results  highlight  the  variability  of  training  outcomes  when  the  estimated  nature 
of  the  model  is  explicitly  considered.  In  cases  where  the  standard  errors  of  the  estimated 
coefficients  are  relatively  large,  we  find  differences  between  the  mean  of  the  simulated 


3  Risk  analysis  was  conducted  for  these  MOS:  15Q,  27M,  33W,  35M,  46Q,  55D,  71D,  74B,  91D,  91W, 
96B,  97E,  and  98C.  Description  of  the  risk  analysis  results  follows  the  main  “discussion”  of  the  MOS. 
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probabilities  and  the  static  prediction  of  the  probability  of  passing  evaluated  at  the  mean 
governing  composite  score.  The  policy  case  impacts  relative  to  the  baseline,  however, 
are  the  same  in  both  simulation  and  static  prediction  modes. 


Overview  of  Regression  Model  Results 

We  estimated  models  for  80  MOS  that  span  18  CMFs.  These  are  shown  in  the  Table  of 
Contents,  where  the  MOS  titles  and  codes  prevailing  during  the  2001  -  2004  period  are 
shown  under  the  current  CMF  structure. 

For  most  of  the  MOS  the  estimated  models  were  statistically  significant.4  In  cases 
where  they  weren’t,  it  was  usually  a  problem  of  an  insufficient  number  of  failure  cases. 
Not  surprisingly,  we  achieved  better  model  fits  for  the  more  “technical”  MOS.  The 
percentage  of  variation  explained  by  the  governing  composite,  demographics,  and 
component  membership  ranged  from  three  to  twenty-one  percent,  and  compares 
favorably  to  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  validation  types  of 
research,  of  which  this  is  a  variant.  5  The  estimated  effect  of  the  governing  composite 
upon  the  probability  of  passing  was  statistically  significant  and  also  of  practical 
importance  in  most  of  the  models.  In  these  models,  we  typically  found  that  the  odds  of 
passing  increased  by  a  factor  of  1.04  to  1.10  for  each  one  point  increase  in  the  composite 
score. 


The  regression  results  are  summarized  in  Table  3  (which  follows  the  Conclusions 
section).6  In  that  table,  for  each  MOS  model  we  report  the  R-square  (goodness-of-fit) 
statistic,  the  estimated  odds  factor  for  the  governing  composite,  the  total  number  of  cases 
in  the  sample,  and  the  number  of  failure  cases.  The  regression  model  estimates  and 
policy  analyses  are  reported  by  MOS  following  these  introductory  and  summary  sections. 

Based  on  the  relatively  broad  sample  of  MOS  examined,  the  usefulness  of  the 
estimated  models  for  conducting  policy  analysis  suggests  three  major  groupings  of  MOS 
(see  Table  2).  In  the  first  group  are  MOS  where  cutoff  score  levels  are  relatively  low  and 
passing  rates  are  uniformly  high.  This  gives  way  to  estimated  composite  effects  that  are 
relatively  low  and  predicted  passing  rates  that  are  uniformly  high  in  the  vicinity  of  the 
current  cutoff  level.  For  these  MOS  there  aren’t  any  striking  policy  issues  because  the 


4  The  model  Chi-square  tests  the  null  hypothesis  that  the  coefficients  for  all  of  the  terms  in  the  model, 
except  the  constant,  are  zero.  This  is  comparable  to  the  overall  F  test  for  regression. 

5  In  terms  of  R-values,  the  correlation  measure  favored  by  psychologists,  the  fits  obtained  range  up  to  0.45, 
and  are  quite  respectable  when  considering  use  of  a  binary  outcome  variable  and  the  implicit  restriction  in 
range  when  estimating  from  an  Army  input  sample. 

6  The  regression  results  have  established  the  tendency  of  predicted  passing  probabilities  to  vary  with 
composite  scores.  Despite  the  generally  moderate  explanatory  power  of  the  regression  models  as 
evidenced  by  goodness-of-fit  statistics,  the  models  are  noticeably  weak  in  predictive  power.  A 
combination  of  factors  seems  to  be  at  work.  In  the  first  place,  the  observed  variation  in  composite  scores  is 
limited  inasmuch  as  Soldiers  have  been  selected  against  the  cutoff  level.  And  second,  there  is  a  paucity  of 
information  available  in  the  data  base  for  use  as  explanatory  variables.  We  really  don’t  have  a  lot  of  insight 
into  why  a  particular  Soldier  passes  and  another  does  not.  Fortunately,  for  the  kinds  of  policy  analyses  that 
we  conduct,  individual  predictive  accuracy  is  not  a  requirement. 
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training  is  apparently  not  demanding,  and  the  estimated  models  reflect  that  reality.7  The 
policy  analyses  for  many  of  these  MOS  suggest  that  cutoff  score  levels  could  be  lowered 
without  paying  a  noticeable  price  in  reduced  passing  rates.  However,  as  noted  above,  if 
the  training  requirements  are  less  demanding  than  the  job  requirements,  then  such 
lowered  cutoff  score  levels  could  be  reflected  in  diminished  job  performance.  In  the 
second  group  are  MOS  where  cutoff  score  levels  are  relatively  high  and  passing  rates  are 
lower,  estimated  composite  effects  are  relatively  high,  and  predicted  passing  rates  are 
more  spread  out  in  the  vicinity  of  the  current  cutoff  level.  For  these  MOS  the  estimated 
models  appear  useful  in  revealing  tradeoffs  as  cutoff  score  levels  are  changed.  In 
addition,  there  is  a  group  of  MOS  that  seems  to  fall  in-between  the  first  and  second 
groupings.  Finally,  there  is  a  third  group  of  MOS  for  which  the  explanatory  power  of  the 
estimated  model  is  low  or  non-existent.  In  most  but  not  all  of  these  the  poor  fit  is 

o 

presumed  due  to  too  few  failure  case  observations  ,  but  a  definitive  conclusion  is  not 
possible. 


Table  2.  Implied  Groupings  of  MOS  for  Policy  Analysis 

First  Group  -  limited  policy 
analysis 

13D,  13E,  13F,  14S,  15P,  21B,  44B, 

45B,  25V,  3 1C,  3  IF,  31U,  52D,  55B, 

63H,  63J,  63W,  67T,  67U,  73C,  71L, 

75B,  75H,  88M,  91E,  91G,  92A,  92F, 

92G,  92Y 

In-Between  Group 

35E,  74B,  91B,  91J,  91R,  91S,  96D,  98K 

Second  Group  -  useful  policy 
analysis 

15B,  15F,  15G,  15Q,  27M,  33W,  35M, 
46Q,  46R,  55D  Phase  I,  71D,  91A,  91C, 
91D,  9 IK,  91P,  91Q,  91T,  91W,  91X, 

96B,  96H,  96U,  97B,  97E,  98C,  98H, 

98J 

Third  Group  -  poor  model  fit 

13B,  13M,  14E,  14J,  25M,  25R,  31L, 

31P,  31R,  31S,  35F,  74C,  91H,  91M, 

98G 

7  There  is  one  MOS  (9 IE)  with  relatively  high  estimated  composite  effects  but  little  variation  in  predicted 
passing  rates  across  policy  changes,  and  thus  belongs  with  the  first  group. 
s  The  apparent  exceptions  are  3 IS  and  98G. 
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Conclusions 


Have  we  identified  a  useful  methodology  that  can  be  used  to  assist  Army  school 
proponents  in  examining  the  appropriateness  of  current  composite  cutoff  score  levels? 
We  believe  we  can  answer  in  the  affirmative  -  at  least  for  the  more  technical  MOS  with 
curricula  that  are  relatively  demanding.  The  ATRRS  data  base  with  its  pass  /  fail 
information  is  readily  available  for  all  MOS,  and  will  support  the  binary  logistic 
regression  method  utilized  here.  Actual  training  performance  scores  would  be  better  for 
analyses  focused  on  ASVAB  validity,  and  they  would  likely  facilitate  better  predictive 
accuracy.  However,  for  the  limited  objectives  of  this  type  of  inquiry,  pass  /  fail  outcome 
measures  may  be  sufficient. 
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Table  3.  Summary  of  Regression  Model  Estimation  Results 

8.17.05 


rev  12.30.05 
rev  3.28.06 

Title 

MOS 

Estimated 

R-square 

Statistic 

Estimated 

Governing 

Composite 

Odds 

Factor 

No. 

Cases 

No. 

Failures 

CMF  13 -Field  Artillery 

Cannon  Crewmember 

13B 

0.008 

1.009 

10779 

148 

FA  Automated  Tactical  Data  Sys  Specialist 

13D 

0.045 

1.047 

1505 

75 

Cannon  Fire  Direction  Specialist 

13E 

0.155 

1.117 

1823 

37 

Fire  Support  Specialist 

13F 

0.074 

1.079 

3800 

108 

MLRS  etc.  Specialist 

13M 

0.003 

1.019 

2635 

34 

CMF  14  -  Air  Defense  Artillery 

Patriot  Fire  Control  Operator  -  Maintainer 

14E 

0.020 

1.020 

728 

46 

Air  Defense  Command  C4I  TOC  Enhanced  Opera 

14J 

0.076 

1.013 

1076 

13 

Avenger  Crewmember 

14S 

0.028 

1.035 

1641 

32 

CMF  15  -  Aviation 

Aircraft  Power  Plant  Repairer 

15B 

0.099 

1.081 

465 

39 

Aircraft  Electrician 

15F 

0.078 

1.080 

417 

42 

Aircraft  Structural  Repairer 

15G 

0.098 

1.073 

490 

124 

Aviation  Operations  Specialist 

15P 

0.051 

1.027 

1338 

89 

Air  Traffic  Control  Operator 

15Q 

0.061 

1.037 

633 

194 

UPI-60  Repairer 

67T 

0.045 

1.041 

1670 

61 

CH-47  Repairer 

67U 

0.041 

1.027 

972 

83 

CMF  21  -  Engineer 

Combat  Engineer 

21 B 

0.056 

1.041 

1902 

68 

CMF  25  -  Communications  and  Information  Systems  Operation 
Information  Systems  Operator  -  Analyst  74B  0.059 

1.048 

2360 

173 

Telecommunications  Operator  -  Maintainer 

74C 

0.049 

1.029 

739 

22 

Multimedia  Illustrator 

25M 

0.009 

1.035 

258 

9 

Visual  Information  Eq  Operator  /  Maintainer 

25R 

0.027 

1.001 

105 

14 

Combat  Documentation  /  Production  Specialist 

25V 

0.098 

1.025 

215 

15 

Radio  Operator  -  Maintainer 

31C 

0.047 

1.057 

1793 

15 

Network  Switching  Systems  Operator 

31F 

0.038 

1.045 

3872 

313 

Cable  Systems  Installer  -  Maintainer 

31 L 

0.014 

1.039 

2131 

15 

Microwave  Systems  Operator  -  Maintainer 

31 P 

0.016 

1.026 

755 

48 

Multichannel  Transmission  Sys  Opr  -  Mtr 

31 R 

0.017 

1.021 

5849 

45 

Satellite  Communications  Specialist 

31S 

0.010 

1.031 

1236 

94 

Signal  Support  Systems  Specialist 

31U 

0.030 

1.036 

3568 

171 

CMF  27  -  Paralegal 

Legal  Specialist 

71 D 

0.162 

1.152 

423 

47 

CMF  33  -  Electronic  Warfare  /  Intercept  Systems  Maintenance 


Military  Intelligence  Systems  Operator  /  Integrator 

33W 

0.151 

1.127 

572 

136 

CMF  42  -  Adjutant  General 

Personnel  Administration  Specialist 

75B 

0.034 

1.029 

2921 

67 

Personnal  Services  Specialist 

75H 

0.059 

1.041 

3024 

67 

Administrative  Specialist 

71 L 

0.038 

1.045 

7525 

36 

CMF  44  -  Financial  Management 

Finance  Specialist 

73C 

0.121 

1.099 

1518 

112 
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CMF  46  -  Public  Affairs 
Public  Affairs  Specialist 
Public  Affairs  Broadcast  Specialist 

CMF  63  -  Mechanical  Maintenance 

Metal  Worker 

Small  Arms  Repairer 

Power  Generation  Equipment  Repairer 

Track  Vehicle  Repairer 

QM  &  Chemical  Equipment  Repairer 

Wheeled  Vehicle  Repairer 

CMF  68  -  Medical 

Medical  Equipment  Repairer 

Medical  Specialist 

Health  Care  -  Practical  Nurse 

Operating  Room  Specialist 

Dental  Specialist 

Patient  Administration  Specialist 

Optical  Laboratory  Specialist 

Medical  Logistics  Specialist 

Medical  Laboratory  Specialist 

Hospital  Food  Service  Specialist 

Radiology  Specialist 

Pharmacy  Specialist 

Veterinary  Food  Inspection  Specialist 

Preventative  Medicine  Specialist 

Animal  Care  Specialist 

Health  Care  Specialist 

Mental  Health  Specialist 

CMF  88  -  Transportation 
Motor  T ransport  Operator 

CMF  89  -  Ammunition 
Ammunition  Specialist 

Explosive  Ordnance  Disposal  Specialist  -  Ph.  I 
Explosive  Ordnance  Disposal  Specialist  -  Ph.  II 

CMF  92  -  Supply  and  Services 
Automated  Logistics  Specialist 
Unit  Supply  Specialist 
Petroleum  Supply  Specialist 
Food  Service  Specialist 

CMF  94  -  Electronic  Maintenance  and  Calibrations 
Radio  /  COMSEC  Repairer 
Special  Electronic  Device  Repairer 
Radar  Repairer 

Multiple  Launch  Rocket  System  Repairer 

CMF  96  -  Military  Intelligence 
Intelligence  Analyst 
Imagery  Analyst 

Common  Ground  Station  Operator 
Unmanned  Aerial  Vehicle  Operator 
Counterintelligence  Analyst 
Human  Intelligence  Collector 


46Q 

0.162 

1.058 

402 

66 

46R 

0.164 

1.097 

259 

66 

44B 

0.065 

1.039 

860 

117 

45B 

0.169 

1.129 

540 

45 

52D 

0.055 

1.060 

3275 

83 

63H 

0.063 

1.033 

1205 

41 

63J 

0.096 

1.054 

1643 

106 

63W 

0.070 

1.036 

4486 

143 

91 A 

0.062 

1.034 

241 

135 

91 B 

0.125 

1.065 

5054 

381 

91C 

0.135 

1.100 

475 

80 

91D 

0.155 

1.103 

1063 

158 

91 E 

0.163 

1.150 

1143 

84 

91G 

0.048 

1.037 

652 

22 

91H 

0.115 

1.147 

86 

10 

91 J 

0.087 

1.072 

709 

78 

91 K 

0.204 

1.146 

123 

23 

91 M 

0.047 

1.015 

837 

11 

91 P 

0.080 

1.089 

814 

65 

91Q 

0.162 

1.087 

572 

129 

91 R 

0.037 

1.075 

507 

28 

91S 

0.076 

1.124 

452 

23 

91T 

0.214 

1.143 

376 

52 

91 W 

0.137 

1.102 

10972 

1565 

91X 

0.161 

1.061 

678 

107 

88M 

0.042 

1.035 

14310 

200 

55B 

0.031 

1.042 

2680 

33 

55D 

0.136 

1.085 

445 

235 

55D 

0.043 

1.046 

178 

119 

92A 

0.070 

1.050 

11458 

359 

92Y 

0.061 

1.042 

9970 

122 

92  F 

0.058 

1.044 

10491 

174 

92G 

0.026 

1.050 

10206 

39 

35E 

0.055 

1.041 

1268 

89 

35F 

0.035 

1.039 

319 

23 

35M 

0.181 

1.202 

245 

9 

27M 

0.164 

1.093 

244 

42 

96B 

0.151 

1.095 

2863 

617 

96D 

0.089 

1.084 

529 

73 

96H 

0.060 

1.072 

521 

36 

96U 

0.286 

1.216 

116 

30 

97B 

0.210 

1.148 

1162 

107 

97E 

0.188 

1.076 

748 

140 
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CMF  98  -  Signals  Intelligence  /  Electronic  Warfare  Operations 


Signals  Intelligence  Analyst 

98C 

0.187 

1.120 

1621 

302 

Voice  Interceptor 

98G 

0.009 

1.005 

1461 

113 

Communications  Interceptor  /  Locator 

98H 

0.050 

1.041 

474 

277 

Electronics  Intelligence  Analyst 

98J 

0.090 

1.084 

503 

57 

Signal  Collector  &  Identifier 

98K 

0.104 

1.123 

374 

36 

Notes:  Shaded  entry  in  the  R-square  column  indicates  that  the  estimated  regression  model 
Chi-square  test  was  not  statistically  significant  at  the  P  =  .05  level;  shaded  entry  in  the 
number  of  failures  column  indicates  less  than  50  failure  cases. 
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MOS  Analyses 

(see  Table  of  Contents  for  order) 


The  analyses  are  presented  as  separate  mini-reports  for  each  MOS.  For  each 
report,  we  followed  these  steps:  (a)  describe  the  size  and  composition  of  the 
estimation  sample;  additional  sampling  statistics  are  also  shown  in  an 
Appendix;  (b)  estimate  a  binary  logistic  regression  equation  and  highlight 
the  results;  (c)  examine  the  explanatory  power  of  the  estimated  model  using 
goodness-of-fit  statistics;  (d)  analyze  the  impact  of  changing  the  cutoff  score 
upon  eligibility  and  the  probability  of  successfully  completing  the  training. 
For  selected  MOS  (see  footnote  3)  we  also  supplement  the  policy  analysis 
with  risk  analysis.  Note  that  policy  recommendations  per  se  are  not  made. 
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13B:  Cannon  Crew  Member 


The  final  sample  included  10,779  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (98.6%)  or  failed  for  academic  reasons  (1.4%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  all  of  the  Soldiers  were  male  as  this  MOS  is  not  open  to  women, 
most  had  a  high  school  diploma  but  not  more  education  (93%,  3%  some  college,  4% 

GED  or  less),  and  most  were  from  Regular  Army  (57%,  43%  National  Guard).  The 
governing  AA  composite,  Field  Artillery  (FA),  for  this  MOS  has  a  cutoff  score  of  93;  the 
sample  mean  is  106.45  (standard  deviation  =  1 1.243).  See  the  Appendix  for  a  description 
of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 


Table  1.  13B:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

12.052** 

Log  Likelihood 

1551.197 

Nagelkerke  R  Square 

.008 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

3.099 

22.167 

15.059*** 

GUARD 

.559 

1.749 

9.530** 

FA 

.009 

1.009 

1.486 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .005 


11 


Table  1  and  indicate  that  a  model,  including  the  FA  composite  and  Army  component 
(GUARD  =  National  Guard),  accounts  for  less  than  one  percent  of  the  variation  in  the 
dependent  variable.  This  model  is  statistically  significant  (y  =  12.052,  P  =  .002),  and  it 
has  almost  no  explanatory  power  (Nagelkerke  R  =  .008). 

There  is  no  statistically  significant  effect  for  the  FA  composite,  though  there  is 
one  for  National  Guard  membership.  The  model  suggests  that  the  FA  composite  score 
does  not  predict  increased  odds  of  passing,  but  there  are  relatively  few  failure 
observations  for  a  definitive  conclusion. 

Policy  Analysis.  Given  the  model’s  lack  of  explanatory  power,  and 
notwithstanding  the  statistically  significant  effect  for  National  Guard  membership,  this 
model  is  not  appropriate  for  conducting  policy  analysis. 
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Appendix:  Soldier  Characteristics  (13B) 


Gender 

Outcome 

Male 

Female  | 

fail 

.0% 

pass 

.0% 

Education  Level  j 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

.4% 

1 .4% 

.9% 

pass 

99.6% 

98.6% 

99.1% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

.0% 

1 .0% 

pass 

100.0% 

99.0% 
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13D:  Field  Artillery  Tactical  Data  Systems  Specialist 


The  final  sample  included  1505  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (95%)  or  failed  for  academic  reasons  (5%)  were 
included  in  the  analysis  sample  of  490  Soldiers  because  we  assume  that  non-completion 
for  non-academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  all  of  the  Soldiers  were  male  as  this  MOS  is  not  open  to  women, 
most  had  a  high  school  diploma  but  not  more  education  (93%,  5%  some  college,  3% 

GED  or  less),  and  the  greatest  number  were  from  Regular  Army  (86%,  14%  National 
Guard).  The  governing  AA  composite,  Field  Artillery  (FA),  for  this  MOS  has  a  cutoff 
score  of  93;  the  sample  mean  is  106.58  (standard  deviation  =  1 1.429).  See  the  Appendix 
for  a  description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 


Table  1.  13D:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

22.394*** 

Log  Likelihood 

573.664 

Nagelkerke  R  Square 

.045 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-1.912 

.148 

2.444 

GUARD 

.570 

1.769 

1.447 

FA 

.046 

1.047 

14.877*** 

*  =  p  <  .05  ***  =  p<.001 

**  =  p  <  .005 


Table  1  and  indicate  that  a  model  including  the  FA  composite,  as  well  as  Army 
component  (GUARD  =  National  Guard),  accounts  for  almost  five  percent  of  the  variation 
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in  the  dependent  variable.  This  model  is  statistically  significant  (x  =  22.394,  P  <  .001), 
but  its  explanatory  power  is  limited  (Nagelkerke  R~  =  .045). 

There  is  a  statistically  significant  effect  for  the  FA  composite.  The  model  suggests 
that  having  a  higher  FA  composite  score  increases  the  odds  of  passing.  At  the  mean  FA 
score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  5%  in  the  odds  of 
passing  the  course,  and  a  five-point  increase  in  FA  would  increase  the  odds  of  passing  by 
29%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  on  the  first  attempt  based  upon 
the  binary  logistic  model.  These  were  calculated  using  the  formula  for  finding  probability 
in  a  binary  logistic  model,  which  is 

q(P’x)  /  (1  +  q<p’x)). 

Because  GUARD  is  not  statistically  significant,  this  analysis  is  confined  to  Soldiers  from 
the  Regular  Army  (though  results  would  be  similar  for  other  demographic  groups  and 
National  Guard  members).  Based  on  the  model,  when  the  cutoff  score  is  at  its  current 
level  (FA  =  93),  regular  Army  Soldiers  with  an  average  FA  score  (FA  =  1 10.58)  have 
approximately  a  97%  chance  of  passing.  Currently,  about  88%  of  Soldiers  are  eligible  for 
MOS  13D  assignment  at  the  current  cutoff  (FA  =  93).  Lowering  the  cutoff  by  five  points 
(FA  =  88)  would  increase  eligibility  by  seven  percentage  points  (to  95%)  and  the  average 
Soldier  who  would  qualify  for  training  would  practically  have  the  same  chance  of  passing 
(96%).  Raising  the  cutoff  score  by  five  points  (FA  =  98),  the  average  Soldier  would  have 
the  same  chance  of  passing  (97%)  but  eligibility  would  fall  by  about  1 1  percentage 
points. 
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Table  2.  13D:  Probability  that  a  Male  Soldier  with  a  High  School  Diploma  (from  the 
larger  Army  contract  population)  will  pass  the  course  based  on  the  binary  logistic 
model 


Cutoff  = 

Cutoff = 

Cutoff = 

Cutoff = 

Cutoff = 

83 

88 

93 

98 

113 

Percent  Eligible 
(Regular  Army) 

98.6% 

94.8% 

87.7% 

76.8% 

61.9% 

Mean 

107.86 

108.80 

110.58 

112.86 

115.61 

Passing  rate: 

High  school  male 

96.1% 

96.4% 

96.8% 

97.3% 

97.8% 

Figure  1  shows  the  relationship  between  FA  and  the  probability  of  passing  for 
Regular  Army  Soldiers  and  Guardsmen,  including  upper  and  lower  bounds  based  upon 
the  standard  error  of  the  estimated  FA  coefficient.  For  a  particular  FA  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
corresponding  probability  of  passing.  For  example,  an  FA  score  of  100  corresponds  to  a 
passing  probability  of  about  95%  for  a  high  school  educated,  Regular  Army  male  Soldier. 
One  can  see  that  Soldiers  with  a  relatively  low  FA  score  have  a  reasonable  chance  of 
passing  the  course.  As  FA  scores  increase,  the  chance  that  a  Soldier  will  pass  the  course 
increases. 
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Figure  1:  Predicted  Probability  of  Passing:  Regular  Army 


FA  composite  score 


| Regular  Army  -  -  -  Army  Lower  — —  Guard  —  -  Guard  Lower  -  -  -  Army  Upper  —  -  Guard  Upper 
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Appendix:  Soldier  Characteristics  (13D) 


Gender 

Male 

Female 

fail 

5.0% 

.0% 

pass 

95.0% 

.0% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

2.6% 

5.2% 

1 .5% 

pass 

97.4% 

94.8% 

98.5% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

5.4% 

.0% 

pass 

94.6% 

.0% 
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13E:  Cannon  Fire  Direction  Specialist 


The  final  sample  included  1 823  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (98.0%)  or  failed  for  academic  reasons  (2.0%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  this  sample,  all  of  the  students  were  male  as  this  MOS  is  not  open  to  women, 
most  had  a  high  school  diploma  but  not  more  education  (90%,  6%  some  college,  4% 

GED  or  less),  and  the  greatest  number  were  from  the  National  Guard  (59%,  41% 
Regular).  The  governing  AA  composite,  Field  Artillery  (FA),  for  this  MOS  has  a  cutoff 
score  of  93;  the  sample  mean  is  108.43  (standard  deviation  =  1 1.979).  See  the  Appendix 
for  a  description  of  Soldier  characteristics  for  this  MOS  training  sample. 


Table  1.  13E:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

51.568*** 

Log  Likelihood 

310.077 

Nagelkerke  R  Square 

.155 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-7.924 

.000 

10.809*** 

GED 

-1.454 

.234 

4.620* 

COLLEGE 

-.749 

.473 

.970 

GUARD 

1.476 

4.374 

12.629*** 

FA 

.111 

1.117 

20.633*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .005 


A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  a  model,  including  the  FA  composite,  education  status  (GED  = 
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GED  or  less  than  a  high  school  diploma,  COLLEGE  =  some  college  or  more  education), 
and  Army  component  (GUARD  =  National  Guard),  accounts  for  about  16  percent  of  the 
variation  in  the  dependent  variable.  This  model  is  statistically  significant  (%2  =  51.568,  P 
<  .001),  and  has  moderate  explanatory  power  (Nagelkerke  R~  =  .155). 

There  is  a  statistically  significant  effect  for  the  FA  composite,  GED,  and 
GUARD.  The  model  suggests  that  having  a  higher  FA  composite  score  increases  the  odds 
of  passing.  At  the  mean  FA  score,  an  increase  of  one  point  is  associated  with  an  increase 
of  about  1 1%  in  the  odds  of  passing  the  course,  and  a  five-point  increase  in  FA  would 
increase  the  odds  of  passing  by  74%.  Having  less  than  a  high  school  diploma  decreases 
the  odds  of  passing  by  77%.  National  Guard  membership  increases  the  odds  of  passing 
by  337%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Regular  Army 
Soldier  from  the  larger  Army  contract  population  would  pass  the  course  based  upon  the 
binary  logistic  model.  These  were  calculated  using  the  formula  for  finding  probability  in 
a  binary  logistic  model,  which  is 

Q(P’X)  /  (1  +  Q(P’x>y 

Because  the  COLLEGE  is  not  significant,  the  following  analysis  will  be  based  upon  Male 
Regular  Army  Soldiers  factored  by  GED.  Based  on  the  model,  when  the  cutoff  score  is  at 
its  current  level  (FA  =  93),  high  school  educated  Soldiers  with  an  average  FA  score  (FA 
=  110.12)  have  approximately  a  99%  chance  of  passing,  while  those  with  less  than  a  high 
school  diploma  have  approximately  a  95%  chance  of  passing.  Currently,  about  88%  of 
Soldiers  are  eligible  for  MOS  13E  assignment  at  the  current  cutoff.  Lowering  the  cutoff 
by  five  points  (FA  =  88)  would  increase  eligibility  by  seven  percentage  points  (to  95%) 
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and  the  average  high  school  educated  Soldier  who  would  qualify  for  training  would 
practically  have  the  same  chance  of  passing  (98%),  as  would  the  average  soldier  with  less 
than  a  high  school  diploma  (94%).  Raising  the  cutoff  score  by  five  points  (FA  =  98),  the 
average  high  school  educated  Soldier  would  have  essentially  the  same  chance  of  passing 
(99%),  as  would  Soldiers  with  less  than  a  highs  school  diploma  (96%).  However, 
eligibility  would  fall  by  about  1 1  percentage  points. 


Table  2.  13E:  Probability  that  a  Regular  Arm 
population)  will  pass  the  course  based  on  the  1 

y  Soldier  (from  the  larger  Army  contract 
binary  logistic  model 

Cutoff  = 

83 

Cutoff = 

88 

Cutoff  = 

93 

Cutoff = 

98 

Cutoff = 
103 

Percent  Eligible 
( Regular  Army) 

98.6 

94.8 

87.7 

76.8 

61.9 

Mean 

107.73 

108.63 

110.12 

112.27 

115.21 

Passing  Rate: 

High  School 
Male 

98.3% 

98.4% 

98.7% 

98.9% 

99.2% 

GED 

Male 

93.0% 

93.6% 

94.5% 

95.6% 

96.8% 

Figure  1  shows  the  relationship  between  FA  and  the  probability  of  passing  for 
high  school  and  GED  male  Regular  Army  Soldiers,  including  upper  and  lower  bounds 
based  upon  the  standard  error  of  the  estimated  FA  coefficient.  For  a  particular  FA  score, 
trace  a  vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to 
determine  the  corresponding  probability  of  passing.  For  example,  an  FA  score  of  100 
corresponds  to  a  passing  probability  of  about  96%  for  a  high  school  educated,  Regular 
Army  male  Soldier.  One  can  see  that  high  school  educated  Soldiers  with  “low”  FA  scores 
have  a  poor  chance  of  passing.  However,  this  chance  increases  rather  sharply  as  scores 
increase.  The  chance  of  passing  for  a  Soldier  with  less  than  a  high  school  diploma 
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follows  a  similar  pattern,  but  the  improvement  in  the  odds  of  passing  takes  place  more 
slowly. 


FA  Composite  Score 


Appendix:  Soldier  Characteristics  (13E) 


Gender 

Outcome 

Male 

Female 

2.0% 

.0% 

98.0% 

.0% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

4.1% 

2.0% 

1 .8% 

95.9% 

98.0% 

98.2% 

Branch  of  Army 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

.0% 

.8% 

pass 

.0% 

99.2% 
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13F:  Fire  Support  Specialist 


The  final  sample  included  3800  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (97.2%)  or  failed  for  academic  reasons  (2.8%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  this  sample,  all  of  the  students  were  male  as  this  MOS  is  not  open  to  women, 
most  had  a  high  school  diploma  but  not  more  education  (92%,  5%  some  college,  3% 

GED  or  less),  and  the  greatest  number  were  from  Regular  Army  (69%,  31%  National 
Guard).  The  governing  AA  composite,  Field  Artillery  (FA),  for  this  MOS  has  a  cutoff 
score  of  96;  the  sample  mean  is  1 1 1 .67  (standard  deviation  =  10.287).  See  the  Appendix 
for  a  description  of  Soldier  characteristics  for  this  MOS  training  sample. 


Table  1.  13F:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

64.804*** 

Log  Likelihood 

917.192 

Nagelkerke  R  Square 

.074 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-4.986 

.007 

14.248*** 

GUARD 

1.227 

3.411 

17.673*** 

FA 

.076 

1.079 

37  742*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .005 


A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  a  model,  including  the  FA  composite  and  Army  component 
(GUARD  =  National  Guard),  accounts  for  about  seven  percent  of  the  variation  in  the 
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dependent  variable.  This  model  is  statistically  significant  (x  =  64.804,  P  <  .001),  and  has 
somewhat  limited  explanatory  power  (Nagelkerke  R“  =  .074). 

There  is  a  statistically  significant  effect  for  the  FA  composite  and  GUARD.  The 
model  suggests  that  having  a  higher  FA  composite  score  increases  the  odds  of  passing. 

At  the  mean  FA  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  8% 
in  the  odds  of  passing  the  course,  and  a  five-point  increase  in  FA  would  increase  the  odds 
of  passing  by  46%.  National  Guard  membership  increases  the  odds  of  passing  by  241%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Regular  Army 
Soldier  from  the  larger  Army  contract  population  would  pass  the  course  based  upon  the 
binary  logistic  model.  These  were  calculated  using  the  formula  for  finding  probability  in 
a  binary  logistic  model,  which  is 

Q(P’x>  /  (1  +  Q(P’x)y 

Based  on  the  model,  when  the  cutoff  score  is  at  its  current  level  (FA  =  96),  Soldiers  with 
an  average  FA  score  (FA  =  1 1 1.41)  have  approximately  a  97%  chance  of  passing. 
Currently,  about  81%  of  Soldiers  are  eligible  for  MOS  13F  assignment  at  the  current 
cutoff  (FA  =  96).  Lowering  the  cutoff  by  five  points  (FA  =  91)  would  increase  eligibility 
by  10  percentage  points  (to  91%)  and  the  average  Soldier  who  would  qualify  for  training 
would  practically  have  the  same  chance  of  passing  (96%).  Raising  the  cutoff  score  by 
five  points  (FA  =  101),  the  average  Soldier  would  have  almost  the  same  chance  of 
passing  (98%)  but  eligibility  would  fall  by  about  12  percentage  points. 
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Table  2.  13F:  Probability  that  a  Regular  Anny  Soldier  (from  the  larger  Army  contract 
population)  will  pass  the  course  based  on  the  binary  logistic  model 

Cutoff  = 
86 

Cutoff = 
91 

Cutoff = 
93 

Cutoff  = 
96 

Cutoff = 
101 

Cutoff = 
106 

Percent  Eligible 
(Regular  Army) 

96.9% 

90.9% 

87.7% 

81.3% 

68.6% 

53.0% 

Mean 

108.37 

109.70 

110.12 

111.94 

114.53 

117.49 

Passing  Rate: 

High  School 
Male 

96.3% 

96.6% 

96.7% 

97.1% 

97.6% 

98.1% 

Figure  1  shows  the  relationship  between  FA  and  the  probability  of  passing  for 
Regular  Army  Soldiers  and  Guardsmen,  including  upper  and  lower  bounds  based  upon 
the  standard  error  of  the  estimated  FA  coefficient.  For  a  particular  FA  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
corresponding  probability  of  passing.  For  example,  an  FA  score  of  100  corresponds  to  a 
passing  probability  of  about  95%  for  a  high  school  educated,  Regular  Army  male  Soldier. 
One  can  see  that  Soldiers  with  “low”  FA  scores  have  a  reasonable  chance  of  passing  the 
course.  As  FA  scores  increase,  the  chance  that  a  Soldier  will  pass  the  course  increases. 
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Figure  1:  Predicted  Probability  of  Passing:  Regular  Army 


Appendix:  Soldier  Characteristics  (13F) 


Gender 

Outcome 

Male 

Female 

2.8% 

.0% 

97.2% 

.0% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

1.9% 

3.0% 

98.1% 

97.0% 

Branch  of  Army 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

3.6% 

pass 

96.4% 
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13M:  Multiple  Launch  Rocket  System  Crew  Member 


The  final  sample  included  2635  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (98.7%)  or  failed  for  academic  reasons  (1.3%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  random  and  not  related  to  cognitive  criteria. 

In  the  sample,  all  of  the  Soldiers  were  male  as  this  MOS  is  not  open  to  women, 
most  had  a  high  school  diploma  but  not  more  education  (95%,  4%  some  college,  1% 

GED  or  less),  and  most  were  from  Regular  Army  (79%,  21%  National  Guard).  The 
governing  AA  composite,  Operator  /  Food  (OF),  for  this  MOS  has  a  cutoff  score  of  100; 
the  sample  mean  is  1 10.54  (standard  deviation  =  8.865).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  a  model,  including  the  OF  composite  and  Army  component 
(GUARD  =  National  Guard),  accounts  for  about  0.3%  of  the  variation  in  the  dependent 


Table  1.  13M:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

1.170 

Log  Likelihood 

362.208 

Nagelkerke  R  Square 

.003 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

2.234 

9.340 

1.010 

GUARD 

.282 

1.326 

.385 

OF 

.019 

1.019 

.851 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .005 
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variable.  This  model  is  not  statistically  significant  (x  =  1.170,  P  =  .557),  and  it  has  no 
explanatory  power  (Nagelkerke  R“  =  .003). 

There  is  no  statistically  significant  effect  for  the  OF  composite  or  National  Guard 
membership. 

Policy  Analysis.  Because  there  are  too  few  failure  cases  and  the  model  for  MOS 
13M  is  not  significant,  policy  analysis  cannot  be  conducted. 
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Appendix:  Soldier  Characteristics  (13M) 


Gender 

Outcome 

Male 

Female  | 

fail 

1.3% 

.0% 

pass 

98.7% 

.0% 

Education  Level  I 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

.0% 

1 .3% 

pass 

100.0% 

98.7% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

1.4% 

.0% 

1.1% 

pass 

98.6% 

.0% 

98.9% 

31 


14E:  Patriot  Fire  Control  Enhanced  Operator-  Maintainer 


The  final  sample  included  728  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  for  this  MOS  during  a  period  from  2001 
to  2004.  Only  Soldiers  who  graduated  (93.7%)  or  failed  for  academic  reasons  (6.3%) 
were  included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (88%,  12%  female),  most  had  a  high  school 
diploma  but  not  more  education  (92%,  6%  some  college,  2%  GED  or  less),  and  most 
were  from  were  from  the  Regular  Anny  (98%,  2%  National  Guard).  The  governing  AA 
composite,  Mechanical  Maintenance  (MM),  for  this  MOS  has  a  cutoff  score  of  102;  the 
sample  mean  is  1 12.77  (standard  deviation  =  8.958).  See  the  Appendix  for  a  description 
of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  binary  logistic  model,  including  the  MM  composite  and 
gender  (GENDER  =  female),  accounts  for  two  percent  of  the  variation  in  the  dependent 


Table  1.  14E:  Results  of  the  forward  stepwise 
binary  logistic  prediction  model _ 


Chi-  Square 

5.652 

Log  Likelihood 

337.451 

Nagelkerke  R  Square 

.021 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

.337 

.802 

1.401 

GENDER 

1.237 

3.447 

2.849 

MM 

.020 

1.020 

2.322 

*  =  p  <  .05  ***  =  p  <  .001 

**  =p  <  .01 
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9 

variable.  This  model  is  not  statistically  significant  (x  =  5.652,  P  =  .059),  and  it  has 

2 

limited  explanatory  power  (Nagelkerke  R“  =  .021). 


There  are  no  statistically  significant  effects  for  the  MM  composite.  The  data  are 
insufficient  (too  few  failure  observations)  to  determine  if  any  of  the  variables,  including 
MM,  have  a  consistent  effect  upon  passing.9 

Policy  Analysis.  Given  the  model’s  lack  of  explanatory  power,  this  model  is  not 
appropriate  for  conducting  policy  analysis. 


9  In  previous  research  [Williams  and  Greenston,  2004]  we  reported  a  statistically  significant  model  (with 
MM,  GENDER,  COLLEGE,  and  GED)  for  this  MOS  with  data  over  the  1999  -  2000  period.  Evidently 
there  are  too  few  failure  cases  and  insufficient  variation  across  the  explanatory  variables  in  the  current  data 
set  (i.e.,  over  the  2001  -  2004  period). 
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Appendix:  Soldier  Characteristics  (14E) 


Gender 

Outcome 

Male 

Female  j 

fail 

6.9% 

2.2% 

pass 

93.1% 

97.8% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

.0% 

6.9% 

.0% 

pass 

100.0% 

93.1% 

100.0% 

Outcome 

Component 

Army 

National 

Regular  Army 

Reserve 

Guard 

6.5% 

.0% 

.0% 

93.5% 

.0% 

100.0% 
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14 J:  Air  Defense  Command  C4I  TOC  Enhanced  Operator/Maintainer 


The  final  sample  included  1076  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (98.8%)  or  failed  for  academic  reasons  (1.2%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  of  the  Soldiers  were  male  (92%,  8%  female),  most  had  a  high 
school  diploma  but  not  more  education  (87%,  10%  some  college,  3%  GED  or  less),  and 
most  were  from  Regular  Army  (88%,  12%  National  Guard).  The  governing  AA 
composites,  General  Technical  (GT)  and  Mechanical  Maintenance  (MM),  for  this  MOS 
have  a  cutoff  scores  of  100  and  97  respectively;  the  sample  mean  for  GT  is  1 12.63 
(standard  deviation  =  9.353)  and  the  sample  mean  for  MM  is  110.13  (standard  deviation 
=  10.391).  See  the  Appendix  for  a  description  of  Soldier  characteristics  for  this  MOS 
training  sample. 


Table  1.  14J:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

10.087 

Log  Likelihood 

130.572 

Nagelkerke  R  Square 

.076 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

4.780 

9.055 

1.534 

GED 

-2.401 

.091 

7.064** 

COLLEGE 

-.785 

.456 

.884 

GUARD 

-.040 

.961 

.002 

GENDER 

-.825 

.438 

1.061 

GT 

-.012 

.988 

1.52 

MM 

.013 

1.013 

.520 

*  =  p  <  .05  ***=p<.001 

**  =  p<  .01 
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A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  a  model,  including  the  GT  and  MM  composites,  education 
status  (GED  =  GED  or  less  education;  COLLEGE  =  some  college  education),  Army 
component  (GUARD  =  National  Guard),  and  gender  (GENDER  =  female),  accounts  for 
about  eight  percent  of  the  variation  in  the  dependent  variable.  This  model  is  not 
statistically  significant  (%2  =  10.087,  P  =  .121). 

There  is  no  statistically  significant  effect  for  the  GT  or  MM  composites,  though 
there  is  one  for  having  a  GED  or  less  education.  The  model  suggests  that  neither  the  GT 
nor  the  MM  composite  scores  predict  increased  odds  of  passing,  but  there  are  relatively 
few  failure  observations  for  a  definitive  conclusion. 

Policy  Analysis.  Given  the  model’s  overall  lack  of  statistical  significance,  and 
notwithstanding  the  statistically  significant  effect  for  education  status,  this  model  is  not 
appropriate  for  conducting  policy  analysis. 
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Appendix:  Soldier  Characteristics  (14J) 


Gender 

Outcome 

Male 

Female  |! 

fail 

1.1% 

pass 

98.9% 

Education  Level  j 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

8.8% 

.9% 

1 .9% 

pass 

91.2% 

99.1% 

98.1% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

1.1% 

.0% 

pass 

98.9% 

.0% 
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14S:  Avenger  Crewmember 


The  final  sample  included  1641  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (98%)  or  failed  for  academic  reasons  (2%)  were 
included  in  the  analysis  sample  of  1641  Soldiers  because  we  assume  that  non-completion 
for  non-academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  all  of  the  Soldiers  were  male  as  this  MOS  is  not  open  to  women, 
most  had  a  high  school  diploma  but  not  more  education  (90%,  3%  some  college,  7% 

GED  or  less),  and  the  greatest  number  were  from  Regular  Army  (69%,  31%  National 
Guard).  The  governing  AA  composite,  Operator  /  Food  (OF),  for  this  MOS  has  a  cutoff 
score  of  85;  the  sample  mean  is  100.39  (standard  deviation  =  1 1.12.220).  See  the 
Appendix  for  a  description  of  Soldier  characteristics  for  this  MOS  training  sample. 


Table  1.  14S:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

7.918* 

Log  Likelihood 

307.380 

Nagelkerke  R  Square 

.0 

128 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

.497 

1.644 

.141 

GUARD 

.441 

1.509 

.763 

OF 

.034 

1.035 

5.895* 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 


A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  a  model  including  the  OF  composite,  as  well  as  Army 
component  (GUARD  =  National  Guard),  accounts  for  almost  three  percent  of  the 
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variation  in  the  dependent  variable.  This  model  is  statistically  significant  fy2  =  7.981,  P  < 

2 

.018),  but  its  explanatory  power  is  limited  (Nagelkerke  R“  =  .028). 

There  is  a  statistically  significant  effect  for  the  OF  composite.  The  model  suggests 
that  having  a  higher  OF  composite  score  increases  the  odds  of  passing.  At  the  mean  OF 
score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  3%  in  the  odds  of 
passing  the  course,  and  a  five-point  increase  in  OF  would  increase  the  odds  of  passing  by 
19%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

Q(P’X>  /  (1  + 

Because  GUARD  is  not  statistically  significant,  this  analysis  is  confined  to  Soldiers  from 
the  Regular  Army  (though  results  would  be  similar  for  other  demographic  groups  and 
National  Guard  members).  Based  on  the  model,  when  the  cutoff  score  is  at  its  current 
level  (OF  =  85),  regular  Army  Soldiers  with  an  average  OF  score  (OF  =  106.76)  have 
approximately  a  96%  chance  of  passing.  Currently,  about  97%  of  Soldiers  are  eligible  for 
MOS  14S  assignment  at  the  current  cutoff  (OF  =  85).  Lowering  the  cutoff  by  five  points 
(OF  =  80)  would  increase  eligibility  by  two  percentage  points  (to  99%)  and  the  average 
Soldier  who  would  qualify  for  training  would  practically  have  the  same  chance  of  passing 
(96%).  Raising  the  cutoff  score  by  five  points  (OF  =  90),  the  average  Soldier  would  have 
the  same  chance  of  passing  (96%)  but  eligibility  would  fall  by  about  five  percentage 
points. 
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Table  2.  14S:  Probabi 
population)  will  pass  t 

lity  that  a  Regular  Anny  Soldier  (from  the  larger  Army  contract 
le  course  based  on  the  binary  logistic  model 

Cutoff  = 

75 

Cutoff = 

80 

Cutoff = 

85 

Cutoff = 

90 

Cutoff = 

95 

Percent  Eligible 
(Regular  Army) 

99.9% 

99.3% 

97.2%> 

91.9% 

80.9% 

Mean 

105.89 

106.10 

106.76 

108.05 

110.08 

Passing  rate: 

High  school  male 

95.5% 

95.6% 

95.8% 

96.2% 

96.7% 

Figure  1  shows  the  relationship  between  OF  and  the  probability  of  passing  for 
male  Regular  Army  Soldiers,  including  upper  and  lower  bounds  based  upon  the  standard 
error  of  the  estimated  OF  coefficient.  For  a  particular  OF  score,  trace  a  vertical  line  up  to 
the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the  corresponding 
probability  of  passing.  For  example,  an  OF  score  of  100  corresponds  to  a  passing 
probability  of  about  98%  for  a  high  school  educated,  Regular  Army  male  Soldier.  One 
can  see  that  Soldiers  with  a  relatively  low  OF  score  have  a  very  high  chance  of  passing 
the  course.  As  OF  scores  increase,  the  chance  that  a  Soldier  will  pass  the  course 
increases. 
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Figure  1:  Predicted  Probability  of  Passing:  Regular  Army  Male 


Appendix:  Soldier  Characteristics  (14S) 


Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

1.8% 

2.0% 

.0% 

pass 

98.2% 

98.0% 

100.0% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

2.3% 

pass 

97.7% 
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15B  (68B):  Aircraft  Power  Plant  Repairer 


The  final  sample  included  465  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (91.6%)  or  failed  for  academic  reasons  (8.4%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (97%,  3%  female),  most  had  a  high  school 
diploma  but  not  more  education  (87%,  6%  some  college,  7%  GED  or  less),  and  most 
were  from  the  Regular  Army  (63%,  34%  Guard,  3%  Reserve).  The  governing  AA 
composite,  Mechanical  Maintenance  (MM),  for  this  MOS  has  a  cutoff  score  of  102;  the 
sample  mean  is  1 16.23  (standard  deviation  =  9.017).  See  the  Appendix  for  a  description 
of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  model,  including  the  MM  composite,  education  status  (GED 
=  GED  or  less  than  a  high  school  diploma),  and  Army  component  (GUARD  =  National 
Guard,  RESERVE  =  Army  Reserve)  accounts  for  about  ten  percent  of  the  variation  in  the 
dependent  variable.  This  model  is  statistically  significant  (x  =  20.689,  P  <  .001),  and  has 
moderate  explanatory  power  (Nagelkerke  R  =  .099). 
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Table  1.  68B:  Results  of  tl 
prediction  model:  MM  on 

re  binary  logistic 
ly  model 

Chi-  Square 

20.689*** 

Log  Likelihood 

247.266 

Nagelkerke  R  Square 

.099 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-6.683 

.001 

7.035** 

GED 

-.812 

.444 

1.270 

GUARD 

1.216 

3.372 

6.148* 

RESERVE 

-.506 

.603 

.378 

MM 

.078 

1.081 

12.050*** 

*  =  p  <  .05  ***=p<.001 

**  =p  <  .01 

There  is  a  statistically  significant  effect  for  the  MM  composite.  The  model 
suggests  that  having  a  higher  MM  composite  score  increases  the  odds  of  passing.  At  the 
mean  MM  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  eight 
percent  in  the  odds  of  passing  the  course,  and  a  five-point  increase  in  MM  would  increase 
the  odds  of  passing  by  48%.  There  is  also  a  significant  effect  for  GUARD  where 
guardsmen  have  237%  increased  odds  of  passing. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  probabilities  were  calculated  using  the  formula  for  finding  probability  in  a 
binary  logistic  model,  which  is 

Q(P’x>  /  (1  +  Q(P’x)y 

Because  GED  and  RESERVE  were  not  statistically  significant,  this  analysis  is  confined 
to  the  modal  demographic — Regular  Army  high  school  educated  male  Soldiers  (though 
results  would  be  similar  for  any  demographic  combination,  except  National  Guardsmen). 
Based  on  the  model,  when  the  cutoff  score  is  at  its  current  levels  (MM  =  102),  male 
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Soldiers  with  a  high  school  diploma  and  an  average  MM  score  (MM  =  1 13.76)  have 
approximately  a  90%  chance  of  passing.  Currently,  about  55%  of  Soldiers  are  eligible  for 
MOS  68B  assignment  at  the  current  cutoff.  Lowering  the  MM  cutoff  by  five  points  (MM 


Table  2.  68B:  Probability  that  a  Soldier  (from  the  larger  Anny  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model 

Cutoff: 
MM  =  92 

Cutoff: 
MM  =  97 

Cutoff: 
MM  =  102 

Cutoff: 
MM  =  107 

Cutoff: 
MM  =  112 

Percent  Eligible 
(Regular  Army) 

78.8% 

67.6% 

55.0% 

42.0% 

29.3%> 

Mean:  MM 

108.60 

111.03 

113.76 

116.80 

120.09 

Passing  rates: 

High  School  Male 

85.7% 

87.8% 

89.9% 

91.9% 

93.6% 

=  97)  would  increase  eligibility  by  13  percentage  points,  while  the  average  male  Soldier 
who  would  qualify  for  training  would  have  a  slightly  lower  chance  of  passing  (88%). 
Raising  the  MM  cutoff  scores  by  live  points  (MM  =  107),  the  average  male  Soldier  who 
would  still  qualify  for  the  MOS  would  have  a  slightly  higher  chance  of  passing  (92%), 
but  fewer  Soldiers  would  be  eligible  (42%). 

Figure  1  shows  the  relationship  between  MM  and  the  probability  of  passing  for 
high  school  educated  male  Soldiers,  including  upper  and  lower  bounds  based  upon  the 
standard  error  of  the  estimated  MM  coefficient.  For  a  particular  MM  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
corresponding  probability  of  passing.  For  example,  a  MM  value  of  100  corresponds  to  a 
passing  probability  of  about  75%  for  a  high  school  educated  male  Soldier. 
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Chance  of  Passing 


Figure  1:  Predicted  Probability  of  Passing:  High  School  Male  Soldiers 


MM  Composite  Scores 


^^^“Male  -  -  -  |\y|a|e  Upper  -  -  -  Male  Lower 
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Appendix  A:  Soldier  Characteristics  (68B) 


Gender 

Outcome 

Male 

Female 

fail 

8.4% 

7.1% 

pass 

91.6% 

92.9% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

9.4% 

8.4% 

7.1% 

pass 

90.6% 

91.6% 

92.9% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

10.2% 

16.7% 

4.4% 

pass 

89.8% 

83.3% 

95.6% 
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15F  (68F):  Aircraft  Electrician 


The  final  sample  included  417  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (89.9%)  or  failed  for  academic  reasons  (10.1  %) 
were  included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (95%,  5%  female),  most  had  a  high  school 
diploma  but  not  more  education  (86%,  8%  some  college,  6%  GED  or  less),  and  most 
were  from  the  Regular  Army  (59%,  39%  Guard,  2%  Reserve).  The  governing  AA 
composite,  Mechanical  Maintenance  (MM),  for  this  MOS  has  a  cutoff  score  of  102;  the 
sample  mean  is  114.11  (standard  deviation  =  8.833).  See  the  Appendix  for  a  description 
of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  model,  including  the  MM  composite,  education  status  (GED 
=  GED  or  less  than  a  high  school  diploma),  Army  component  (GUARD  =  National 
Guard),  and  gender  (GENDER  =  Female)  accounts  for  about  eight  percent  of  the 
variation  in  the  dependent  variable.  This  model  is  statistically  significant  (%2  =  15.838,  P 
=  .003),  and  has  somewhat  limited  explanatory  power  (Nagelkerke  R"  =  .078). 
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Table  1.  68F:  Results  of  tl 
prediction  model:  MM  on 

ie  binary  logistic 
ly  model 

Chi-  Square 

15.838** 

Log  Likelihood 

256.597 

Nagelkerke  R  Square 

.078 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-6.205 

.002 

5.833* 

GED 

-.092 

.912 

.018 

GUARD 

-.501 

.606 

2.093 

GENDER 

-.314 

.730 

.273 

MM 

.077 

1.080 

10.975*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =p  <  .01 

There  is  a  statistically  significant  effect  for  the  MM  composite.  The  model 
suggests  that  having  a  higher  MM  composite  score  increases  the  odds  of  passing.  At  the 
mean  MM  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  eight 
percent  in  the  odds  of  passing  the  course,  and  a  five-point  increase  in  MM  would  increase 
the  odds  of  passing  by  47%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  probabilities  were  calculated  using  the  formula  for  finding  probability  in  a 
binary  logistic  model,  which  is 

e(p'x)  /  (1  +  q(P’x)). 

Because  GED,  GUARD,  and  GENDER  were  not  statistically  significant,  this  analysis  is 
confined  to  the  modal  demographic — Regular  Army  high  school  educated  male  Soldiers 
(though  results  would  be  similar  for  any  demographic  combination).  Based  on  the  model, 
when  the  cutoff  score  is  at  its  current  level  (MM  =  102),  male  Soldiers  with  a  high  school 
diploma  and  an  average  MM  score  (MM  =  1 13.76)  have  approximately  a  93%  chance  of 
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passing.  Currently,  about  55%  of  Soldiers  are  eligible  for  MOS  68F  assignment  at  the 
current  cutoff.  Lowering  the  MM  cutoff  by  five  points  (MM  =  97)  would  increase 
eligibility  by  13  percentage  points,  while  the  average  male  Soldier  who  would  qualify  for 
training  would  have  a  slightly  lower  chance  of  passing  (91%).  Raising  the  MM  cutoff 
scores  by  five  points  (MM  =  107),  the  average  male  Soldier  who  would  still  qualify  for 
the  MOS  would  have  a  slightly  higher  chance  of  passing  (94%),  but  fewer  Soldiers  would 
be  eligible  (42%). 


Table  2.  68F:  Probability  that  a  Soldier  (from  the  larger  Army  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model _ _ _ 


Cutoff: 

Cutoff: 

Cutoff: 

Cutoff: 

Cutoff: 

MM  =  92 

MM  =  97 

MM  =  102 

MM  =  107 

MM  =  112 

Percent  Eligible 
(Regular  Army) 

78.8% 

67.6% 

55.0% 

42.0% 

29.3%> 

Mean:  MM 

108.60 

111.03 

113.76 

116.80 

120.09 

Passing  rates: 

High  School  Male 

89.6% 

91.2% 

92.8% 

94.2% 

95.4% 

Figure  1  shows  the  relationship  between  MM  and  the  probability  of  passing  for 
high  school  educated  male  Soldiers,  including  upper  and  lower  bounds  based  upon  the 
standard  error  of  the  estimated  MM  coefficient.  For  a  particular  MM  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
corresponding  probability  of  passing.  For  example,  a  MM  value  of  100  corresponds  to  a 
passing  probability  of  about  81%  for  a  high  school  educated  male  Soldier. 
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Chance  of  Passing 


Figure  1:  Predicted  Probability  of  Passing:  High  School  Male  Soldiers 


MM  Composite  Scores 


^^^“Male  -  -  -  |\y|a|e  Upper  -  -  -  Male  Lower 
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Appendix  A:  Soldier  Characteristics  (68F) 


Gender 

Outcome 

Male 

Female 

fail 

9.6% 

18.2% 

pass 

90.4% 

81.8% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

12.0% 

10.6% 

12.4% 

pass 

88.0% 

89.4% 

87.6% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

8.5% 

90.0% 

12.4% 

pass 

91.5% 

10.0% 

87.6% 
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15G:  Aircraft  Structural  Repairer 


The  final  sample  included  490  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (74.7%)  or  failed  for  academic  reasons  (25.3%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (95%,  5%  female),  most  had  a  high  school 
diploma  but  not  more  education  (89%,  8%  some  college,  3%  GED  or  less),  and  the 
greatest  number  were  from  Regular  Army  (72%,  25%  National  Guard,  3%  Reserve).  The 
governing  AA  composite,  Mechanical  Maintenance  (MM),  for  this  MOS  has  a  cutoff 
score  of  102;  the  sample  mean  is  1 14.63  (standard  deviation  =  8. 1).  See  the  Appendix  for 
a  description  of  Soldier  characteristics  for  this  MOS  training  sample. 


Table  1.  15G:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

33.704*** 

Log  Likelihood 

520.655 

Nagelkerke  R  Square 

.098 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-7.052 

.001 

16.899 

GED 

-.613 

.542 

1.041 

COLLEGE 

.327 

1.387 

.526 

RESERVE 

-.019 

.981 

.001 

GUARD 

.682 

1.977 

5.356* 

GENDER 

.650 

1.915 

1.479 

MM 

.070 

1.073 

21.232*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .005 


A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
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Table  1  and  indicate  that  the  model,  including  the  MM  composite,  education  level  (GED 
=  GED  or  less  than  high  school  diploma;  COLLEGE  =  some  college),  Army  component 
(GUARD  =  National  Guard;  RESERVE  =  Army  Reserve)  and  gender  (GENDER  = 
female),  accounts  for  about  10%  of  the  variation  in  the  dependent  variable.  This  model  is 
statistically  significant  (y2  =  33.704,  P  <  .001),  and  has  moderate  explanatory  power 
(Nagelkerke  R2  =  .098). 

There  are  statistically  significant  effects  for  the  MM  composite  and  National 
Guard  membership.  The  model  suggests  that  having  a  higher  MM  composite  score 
increases  the  odds  of  passing.  At  the  mean  MM  score,  an  increase  of  one  point  is 
associated  with  an  increase  of  about  7%  in  the  odds  of  passing  the  course,  and  a  five- 
point  increase  in  MM  would  increase  the  odds  of  passing  by  41%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

Q(P’X>  /  (1  +  Q(P’*)y 

Because  GED,  COLLEGE,  RESERVE,  and  GENDER  were  not  statistically  significant, 
this  analysis  is  confined  to  male  Soldiers  with  a  high  school  diploma  from  the  Regular 
Army  (though  results  would  be  similar  for  other  demographic  groups).  Based  on  the 
model,  when  the  cutoff  score  is  at  its  current  level  (MM  =  102),  regular  Army  Soldiers 
with  an  average  MM  score  (MM  =  1 14.35)  have  approximately  a  72%  chance  of  passing. 
Currently,  about  56%  of  Soldiers  are  eligible  for  MOS  15G  assignment  at  the  current 
cutoff  (MM  =  102).  Lowering  the  cutoff  by  five  points  (MM  =  97)  would  increase 
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eligibility  by  13  percentage  points  (to  69%),  but  the  average  Soldier  who  would  qualify 
for  training  would  have  a  somewhat  lower  chance  of  passing  (68%).  Raising  the  cutoff 
score  by  five  points  (MM  =  107)  would  lead  to  a  somewhat  higher  chance  of  passing  for 


Table  2.  15G:  Probability  that  a  Soldier  (from  the  larger  Army  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model 

Cutoff  = 

92 

Cutoff = 

97 

Cutoff = 
102 

Cutoff = 
107 

Cutoff = 
112 

Percent  Eligible 
(Regular  Army) 

80.4% 

68.6% 

55.7% 

42.7% 

30.5% 

Mean 

109.05 

111.56 

114.35 

117.44 

120. 78 

Passing  rate: 

High  school  male 

64.1% 

68.1% 

72.2% 

76.3% 

80.3% 

the  average  Soldier  (76%),  but  eligibility  would  fall  by  about  13  percentage  points. 

Figure  1  shows  the  relationship  between  MM  score  and  the  probability  of  passing 
for  Regular  Army  Soldiers  and  Guardsmen,  including  upper  and  lower  bounds  based 
upon  the  standard  error  of  the  estimated  MM  coefficient.  For  a  particular  MM  score,  trace 
a  vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  detennine  the 
corresponding  probability  of  passing.  For  example,  an  MM  value  of  100  corresponds  to  a 
passing  probability  of  about  50%  for  a  high  school  educated,  Regular  Army  male  Soldier. 
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ty  of  pasing 


Figure  1:  Predicted  Probability  of  Passing:  Regular  Army 


MM  composite  score 


Appendix:  Soldier  Characteristics  (15G) 


Gender 

Male 

Female 

Outcome 

fail 

25.6% 

20.0% 

pass 

74.4% 

80.0% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

Outcome 

fail 

29.4% 

25.7% 

18.9% 

pass 

70.6% 

74.3% 

81.1% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

Outcome 

fail 

28.3% 

26.7% 

16.4% 

pass 

71.7% 

73.3% 

83.6% 
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15P:  Aviation  Operations  Specialist 


The  final  sample  included  1338  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (93.3%)  or  failed  for  academic  reasons  (6.7%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  Soldiers  were  male  (62%,  38%  female),  most  had  a  high 
school  diploma  but  not  more  education  (86%,  10%  some  college,  4%  GED  or  less),  and 
the  majority  were  from  Regular  Army  (64%,  32%  National  Guard,  4%  Reserve).  The 
governing  AA  composite,  Skilled  Technical  (ST),  for  this  MOS  has  a  cutoff  score  of  92; 
the  sample  mean  is  105.29  (standard  deviation  =  10.2).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 


Table  1.  15P:  Results  of  the  binary  logistic 
prediction  model 

Chi-  Square 

26.780*** 

Log  Likelihood 

627.597 

Nagelkerke  R  Square 

.051 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-.465 

.628 

.109 

COLLEGE 

.249 

1.283 

.317 

GUARD 

1.201 

3.324 

11.852*** 

GENDER 

.238 

1.269 

1.014 

ST 

.027 

1.027 

3.794* 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p<  .005 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  a  model  including  the  ST  composite,  education  status  (College 


58 


=  some  college  education),  Army  component  (GUARD  =  National  Guard),  and  gender 
(GENDER  =  female)  accounts  for  about  five  percent  of  the  variation  in  the  dependent 
variable.  This  model  is  statistically  significant  (x  =  26.780,  P  <  .001),  but  has  limited 
explanatory  power  (Nagelkerke  R“  =  .05 1). 

There  is  a  statistically  significant  effect  for  the  ST  composite.  The  model 
suggests  that  having  a  higher  ST  composite  score  increases  the  odds  of  passing.  At  the 
mean  ST  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  3%  in  the 
odds  of  passing  the  course,  and  a  five-point  increase  in  ST  would  increase  the  odds  of 
passing  by  14%.  There  is  also  a  significant  effect  for  National  Guard  membership,  where 
Guardsmen  have  232%  better  odds  of  passing  the  course. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

/  (1  +  Q(P’x)y 

Because  GENDER  was  not  statistically  significant,  this  analysis  is  confined  to  male 
Soldiers  (though  results  would  be  similar  for  female  soldiers).  Based  on  the  model,  when 
the  cutoff  score  is  at  its  current  level  (ST  =  92),  male  Soldiers  with  an  average  ST  score 
(ST  =  109.45)  have  approximately  a  92%  chance  of  passing.  Currently,  about  89%  of 
Soldiers  are  eligible  for  MOS  15P  assignment  at  the  current  cutoff  (ST  =  92).  Lowering 
the  cutoff  by  five  points  (ST  =  87)  would  increase  eligibility  by  8  percentage  points  (to 
97%)  and  the  average  Soldier  who  would  qualify  for  training  would  have  approximately 
the  same  probability  of  passing  (92%).  Raising  the  cutoff  score  by  five  points  (ST  =  97) 
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would  leave  the  average  Soldier  with  about  the  same  chance  of  passing  (93%),  but 
eligibility  would  fall  by  about  1 1  percentage  points. 


Table  2.  15P:  Probability  that  a  Soldier  (from  the  larger  Army  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model _ _ _ 


Cutoff  = 

Cutoff = 

Cutoff = 

Cutoff = 

Cutoff = 

82 

87 

92 

97 

102 

Percent  Eligible 
(Regular  Army) 

99.8% 

97.1% 

89.0% 

78.2% 

64.9%o 

Mean 

106.82 

107.67 

109.45 

111.59 

114.16 

Passing  rate: 

Male 

91.8% 

92.0% 

92.3% 

92.7% 

93.2% 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
Regular  Army,  male  Soldiers,  including  upper  and  lower  bounds  based  upon  the  standard 
error  of  the  estimated  ST  coefficient.  For  a  particular  ST  score,  trace  a  vertical  line  up  to 
the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the  corresponding 
probability  of  passing.  For  example,  an  ST  score  of  100  corresponds  to  a  passing 
probability  of  about  90%  for  a  Regular  Army  male  Soldier.  One  can  see  that  Soldiers 
with  a  relatively  low  ST  score  have  a  reasonably  high  chance  of  passing  the  course.  As 
ST  scores  increase,  the  chance  that  a  soldier  will  pass  the  course  increases. 
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Probability  of  pasing 


Figure  1:  Predicted  Probability  of  Passing:  Male 


ST  composite  score 


^^^^Male  -  -  -  Male  Lower  -  -  -  Male  Upper 
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Appendix:  Soldier  Characteristics  (15P) 


Gender 

Outcome 

Male 

Female  | 

fail 

6.9% 

6.3% 

pass 

93.1% 

93.7% 

Education  Level  I 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

.0% 

7.2% 

4.6% 

pass 

100.0% 

92.8% 

95.4% 

Branch  of  Army 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

9.2% 

.0% 

pass 

90.8% 

100.0% 
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15Q:  Air  Traffic  Controller 


The  final  sample  included  633  soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  training  attempt  in  this  MOS  during  a  period  from 
2001  to  2004.  Only  soldiers  who  graduated  (69.4%)  or  failed  for  academic  reasons 
(30.6%)  were  included  in  the  analysis  sample  because  we  assume  that  non-completion  for 
non-academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (74%,  26%  female),  most  had  a  high  school 
diploma  but  not  more  education  (82%,  16%  some  college,  2%  GED  or  less),  and  most 
were  Regular  Army  (87%,  13%  National  Guard).10  The  governing  AA  composite, 

Skilled  Technical  (ST),  for  this  MOS  has  a  cutoff  score  of  102;  the  sample  mean  is 
1 12.75  (standard  deviation  =  10.532).  See  the  Appendix  for  a  description  of  Soldier 
characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  for  this  MOS  and  focused  on 
the  first  training  attempt.  The  estimation  results  are  reported  in  Table  1  and  indicate  that 
the  model,  including  the  ST  composite,  education  status  (GED  =  GED  or  less  than  high 
school  diploma;  COLLEGE  =  some  college),  Army  component  (GUARD  =  National 
Guard)  and  gender  (GENDER  =  female),  accounts  for  about  six  percent  of  the  variation 
in  the  dependent  variable.  This  model  is  statistically  significant  (x  =  27.954,  P  <  .001), 
but  its  explanatory  power  appears  somewhat  limited  (Nagelkerke  R  =  .061). 


10  Four  Soldiers  were  members  of  the  Army  Reserve;  these  Soldiers  were  included  with  the  Regular  Army 
for  data  analysis. 
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Table  1.  15Q:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

27.954*** 

Log  Likelihood 

752.222 

Nagelkerke  R  Square 

.061 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-3.422 

.033 

7.779 

GED 

-.604 

.547 

.970 

COLLEGE 

.459 

1.583 

2.824 

GUARD 

.869 

2.383 

6.927** 

GENDER 

-.081 

.922 

.156 

ST 

.036 

1.037 

11.382** 

**  =  p  <  .005 

***  =  p<  .001 


There  are  statistically  significant  effects  for  both  the  ST  composite  and  National 
Guard  membership.  The  model  suggests  that  having  a  higher  ST  composite  score 
increases  the  odds  of  passing.  At  the  mean  ST  score,  an  increase  of  one  point  is 
associated  with  an  increase  of  about  four  percent  in  the  odds  of  passing  the  course,  and  a 
five-point  increase  in  ST  would  increase  the  odds  of  passing  by  20%.  There  is  also  a 
noteworthy  effect  for  GUARD,  where  the  odds  of  a  Guardsman  passing  this  training 
exceed  that  of  the  average  Soldier  by  138%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

Q(P'x>  /  (1  + 

Because  GENDER,  GED,  and  COLLEGE  were  not  statistically  significant,  this  analysis 
is  confined  to  the  modal  demographics — male  Soldiers  with  a  high  school  diploma 
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(though  results  would  be  similar  for  any  demographic  combination).  Based  on  the 
model,  when  the  cutoff  score  is  at  its  current  level  (ST  =  102),  regular  Anny  soldiers  with 
an  average  ST  score  (ST  =  1 14.16)  have  approximately  a  67%  chance  of  passing. 
Currently,  about  65%  of  Soldiers  are  eligible  for  MOS  15Q  assignment  at  the  current 
cutoff  (ST  =  102).  Lowering  the  cutoff  score  by  five  points  (ST  =  97)  would  increase 
eligibility  by  about  13  percentage  points,  and  the  average  Soldier  who  would  qualify 
would  have  a  slightly  lower  chance  of  passing  (65%).  Raising  the  cutoff  by  five  points 


Table  2.  15Q:  Probability  that  a  Soldier  (from  the  larger  Army  contract 
population)  will  pass  the  course  based  on  the  binary  logistic  model 

Cutoff  = 
92 

Cutoff = 
97 

Cutoff = 
102 

Cutoff = 
107 

Cutoff = 
112 

Percent  Eligible 
( Regular  Army) 

89.0% 

78.2% 

64.9%o 

49. 6%> 

34. 7%> 

Mean 

109.45 

111.59 

114.16 

117.18 

120.51 

Passing  rate: 

Regular  Anny 

62.7% 

64.5% 

66.5% 

68.9% 

71.4% 

National  Guard 

80.0% 

81.2% 

82.6% 

84.1% 

85.6% 

(ST  =  107)  would  lead  to  only  a  slightly  higher  chance  of  passing  (to  69%)  for  the 
average  Soldier,  while  reducing  eligibility  by  15  percentage  points  (to  50%). 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
Regular  Army  Soldiers,  including  upper  and  lower  bounds  based  upon  the  standard  error 
of  the  estimated  ST  coefficient.  For  a  particular  ST  score,  trace  a  vertical  line  up  to  the 
curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the  corresponding 
probability  of  passing.  For  example,  an  ST  score  of  100  corresponds  to  a  passing 
probability  of  about  55%  for  a  high  school  educated,  Regular  Army  male  Soldier.  One 
can  see  that  the  probability  of  passing  increases  gradually,  in  a  near  linear  fashion,  within 
the  range  of  possible  ST  scores. 
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probability  of  passing 


Figure  1:  Predicted  Probability  of  Passing:  Male  High  School  Graduate,  Regular  Army 


ST  composite  score 


^^^“Mean  —  —Upper  Bound  — *  —Lower  Bound 
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Appendix:  Soldier  Characteristics  (15Q) 


Gender 

Male 

Female 

fail 

pass 

29.7% 

70.3% 

33.5% 

66.5% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

pass 

35.7% 

64.3% 

32.2% 

67.8% 

22.0% 

78.0% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

pass 

32.4% 

67.6% 

.0% 

100.0% 

19.2% 

80.8% 
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15Q  Risk  Analysis  of  Policy  Alternatives 


In  this  section  we  extend  the  policy  analysis  with  a  risk  analysis  of  the  policy 
alternatives  considered  using  simulation  methods.  We  employ  the  “@RISK”  software 
package  which  is  designed  as  an  add-in  for  Microsoft  Excel.1 1  There  are  four  steps  in  the 
risk  analysis. 

The  first  step  is  developing  a  model.  A  regression  model  for  predicting  the 
passing  rate  has  been  specified  and  estimated,  and  the  results  stored  in  Excel  worksheet 
format.  See  Table  1. 

The  second  step  is  identifying  the  uncertainty  inherent  in  both  the  input  variables 
(composite  scores,  Soldier  characteristics)  and  the  corresponding  estimated  parameters 
and  specifying  their  possible  values  with  probability  distributions. 

To  approximate  the  governing  composite  distribution,  we  use  the  larger  Army 
contract  population  (EAF  file:  those  who  contracted  from  Jan  1992  to  Aug  2003).  This 
distribution  can  be  assumed  to  follow  a  normal  distribution.  The  mean  and  standard 
deviation  estimated  from  the  contract  population  and  used  as  input  to  the  simulations  are 
shown  in  Table  R-l. 


Table  R-l.  Input  information  for  the  simulations 


Mean 

Std  error  / 
deviation 

Constant 

-3.422 

1.227 

ST  var 

107.02  (before  truncation) 

12.157 

ST  coeff 

.036 

.011 

GENDER  var 

25.7%  (female) 

GENDER  coeff 

.869 

.205 

GED  var 

2.8% 

GED  coeff 

-.604 

.613 

COLLEGE  var 

15.8% 

COLLEGE  coeff 

.459 

.273 

GUARD  var 

12.3% 

GUARD  coeff 

.869 

.330 

1 1  Software  is  available  from  Palisade  Corporation,  Newfield,  NY. 
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To  approximate  the  distributions  for  the  demographic  variables  (GENDER,  GED, 
COLLEGE,  RESERVE,  and  GUARD)  we  rely  on  the  training  data  sample  used  to 
estimate  the  regression  model.  These  distributions  can  be  assumed  to  follow  binomial 
distributions  with  means  estimated  from  the  training  data  sample  as  shown  in  Table  R-l. 

To  approximate  the  distributions  for  the  estimated  parameters  in  the  logit  model, 

we  note  that,  under  very  general  conditions,  the  estimates  will  be  asymptotically  normally 
12 

distributed.  "  The  estimated  coefficients  (means  and  standard  errors)  are  shown  in  Table 
R-l. 


The  third  step  is  analyzing  the  model  with  simulation  to  determine  the  range  and 
probabilities  of  all  possible  training  outcome  results.  The  results  reported  represent  the 

IT 

simulation  of  the  regression  prediction  equation  for  1000  iterations  using  a  fixed  seed. 

The  comparisons  are  between  the  baseline  case  (ST  =  102)  and  the  policy  cases  (ST  =  97 
and  ST  =  107).  These  cases  are  delineated  by  the  cutoff  score  level  which  serves  as  the 
lower  truncation  point  in  the  composite  score  input  distribution.  The  simulation  results  in 
Table  R-2  describe  the  effect  on  the  mean  probability  of  passing  as  the  ST  cutoff  score 
level  is  varied,  and  represent  an  improvement  of  the  policy  analysis  presented  in  Table  2 
because  they  reflect  the  simultaneous  effects  of  all  known  sources  of  uncertainty. 

Table  R-2.  15Q  simulation  results 


ST  =  97 

ST  =  102 

GO 

H 

II 

0 

--4 

Mean  probability 
of  passing 

64.51% 

65.37% 

67.61% 

Std  deviation 

25.93% 

29.13% 

27.82% 

The  frequency  distributions  of  simulated  effects  are  shown  below  in  the  histogram 
graphs.  The  histogram  portrays  the  full  range  of  outcomes  that  average  to  the  mean 
effect  as  reported.  Looking  at  the  histogram  for  the  baseline  case  (ST  =  102),  probability 
of  passing  is  plotted  on  the  horizontal  axis  and  the  height  of  each  bar  indicates  the 
relative  number  of  Soldiers  at  that  passing  probability.  The  information  in  the  histograms 
is  effectively  summarized  in  Table  R-3.  In  that  table  we  see  the  percentage  of  Soldiers 
that  meet  or  exceed  a  given  chance  of  passing.  For  example,  looking  at  the  baseline  case 
(ST  =  102),  just  over  52%  of  Soldiers  have  a  70%  or  greater  chance  of  passing. 


12  The  intuition  is  that  logit  estimates  are  derived  from  maximum  likelihood  (ML)  estimation,  and  under 
fairly  general  conditions  ML  estimates  are  consistent  and  asymptotically  normally  distributed. 

13  Latin  Hypercube  (LH)  sampling  is  utilized  in  drawing  samples  for  the  input  distributions.  It  is  designed 
to  accurately  recreate  the  input  distribution  through  sampling  in  fewer  iterations  when  compared  with  the 
Monte  Carlo  method.  The  key  to  LH  sampling  is  stratification  of  the  input  probability  distributions. 
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Table  R-3. 

Percent  of  simulated  Soldiers  that  meet  or  exceed 
given  chance  of  passing 


Chance  of 
passing 

ST  =  97 

ST  =  102 

ST  =  107 

95.0% 

12.3% 

17.8% 

19.1% 

90.0% 

20.4% 

28.3% 

29.3% 

85.0% 

25.8% 

36.2% 

35.0% 

80.0% 

35.0% 

41.5% 

46.1% 

75.0% 

39.2% 

45.6% 

50.4% 

70.0% 

50.1% 

52.4% 

54.9% 

65.0% 

52.6% 

56.3% 

60.0% 

60.0% 

63.1% 

62.2% 

64.5% 

55.0% 

67.2% 

65.1% 

66.6% 

50.0% 

68.6% 

67.3% 

69.9% 

We  find  that  the  mean  probability  of  passing  in  the  simulation  is  65%  (Table  R-2, 
baseline).  Looking  at  the  histograms,  we  see  the  largest  portion  of  Soldiers  at  the  high 
end  of  passing  probabilities,  and  the  remainder  distributed  across  the  middle  and  lower 
range.  We  note  that  the  mean  simulated  passing  probability  turns  out  to  be  about  the 
same  as  the  static  prediction  at  the  mean  ST  score  (Table  2).  In  the  simulated  predictions 
we  are  calculating  an  average  over  all  Soldiers  and  taking  into  account  all  sources  of 
uncertainty,  whereas  in  the  static  prediction  we  are  calculating  the  probability  of  passing 
only  at  the  mean  ST  score,  ignoring  the  distribution  of  Soldier  scores  etc.  as  well  as 
uncertainty.  While  recognizing  this  variability  is  important,  we  also  note  that  the  policy 
case  impacts  relative  to  the  baseline  are  the  same  in  both  simulation  and  static  prediction 
modes. 


The  fourth  step  is  making  a  decision  based  on  the  results  provided  and  the 
preferences  of  the  school  proponent.  This  involves  the  evaluation  by  the  school 
proponent  of  the  tradeoff  between  increased  eligibility  and  reduced  passing  probabilities 
(and  vice-versa). 
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Distribution  for  PASS  EQUATION  102 


X  <=0.1  X  <=0.99 


Distribution  for  PASS  EQUATION  107 


X  <=0.12  X  <=0.99 
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Distribution  for  PASS  EQUATION  97 


X  <=0.16  X  <=0.98 
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15T  (67T):  UH-60  Repairer 


The  final  sample  included  1670  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (96.3%)  or  failed  for  academic  reasons  (3.7%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (94%,  6%  female),  most  had  a  high  school 
diploma  but  not  more  education  (88%,  4%  some  college,  8%  GED  or  less),  and  most 
were  from  the  Regular  Army  (73%,  25%  Guard,  2%  Reserve).  The  governing  AA 
composite,  Mechanical  Maintenance  (MM),  for  this  MOS  has  a  cutoff  score  of  102;  the 
sample  mean  is  1 16.07  (standard  deviation  =  9. 109).  See  the  Appendix  for  a  description 
of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  model,  including  the  MM  composite,  education  status 
(COLLEGE  =  some  college  education),  Army  component  (GUARD  =  National  Guard, 
RESERVE  =  Army  Reserve),  and  gender  (GENDER  =  female),  accounts  for  about  five 
percent  of  the  variation  in  the  dependent  variable.  This  model  is  statistically  significant 
(X  =  20.521,  P  =  .001),  and  has  limited  explanatory  power  (Nagelkerke  R  =  .045). 
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Table  1.  67T:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

20.521*** 

Log  Likelihood 

503.008 

Nagelkerke  R  Square 

.045 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-1.477 

.228 

.908 

COLLEGE 

.449 

1.568 

.543 

GUARD 

.897 

2.453 

4.809* 

RESERVE 

-.197 

.822 

.069 

GENDER 

-.461 

.631 

1.021 

MM 

.040 

1.041 

8.679** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =p  <  .01 


There  is  a  statistically  significant  effect  for  the  MM  composite.  The  model 
suggests  that  having  a  higher  MM  composite  score  increases  the  odds  of  passing.  At  the 
mean  MM  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  four 
percent  in  the  odds  of  passing  the  course,  and  a  five-point  increase  in  MM  would  increase 
the  odds  of  passing  by  22%.  There  is  also  a  significant  effect  for  GUARD  where 
guardsmen  have  a  145%  increased  odds  of  passing. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  probabilities  were  calculated  using  the  formula  for  finding  probability  in  a 
binary  logistic  model,  which  is 

Q(P’x)  /  (1  + 

Because  COLLEGE,  RESERVE,  and  GENDER  were  not  statistically  significant,  this 
analysis  is  confined  to  the  modal  demographic — Regular  Army  high  school  educated 
male  Soldiers  (though  results  would  be  similar  for  any  demographic  combination,  except 
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National  Guardsmen).  Based  on  the  model,  when  the  cutoff  score  is  at  its  current  level 
(MM  =  102),  male  Soldiers  with  a  high  school  diploma  and  an  average  MM  score  (MM  = 
1 13.76)  have  approximately  a  96%  chance  of  passing.  Currently,  about  55%  of  Soldiers 
are  eligible  for  MOS  67T  assignment  at  the  current  cutoff.  Lowering  the  MM  cutoff  by 


Table  2.  67T:  Probability  that  a  Soldier  (from  the  larger  Army  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model _ _ _ 


Cutoff: 

Cutoff: 

Cutoff: 

Cutoff: 

Cutoff: 

MM  =  92 

MM  =  97 

MM  =  102 

MM  =  107 

MM  =  112 

Percent  Eligible 
(Regular  Army) 

78.8% 

67.6% 

55.0% 

42.0%o 

29.3%, 

Mean:  MM 

108.60 

111.03 

113.76 

116.80 

120.09 

Passing  rates: 

High  School  Male 

94.6% 

95.1% 

95.6% 

96.1% 

96.5% 

five  points  (MM  =  97)  would  increase  eligibility  by  13  percentage  points,  while  the 
average  male  Soldier  who  would  qualify  for  training  would  have  essentially  the  same 
chance  of  passing  (95%).  Raising  the  MM  cutoff  scores  by  five  points  (MM  =  107),  the 
average  male  Soldier  who  would  still  qualify  for  the  MOS  would  have  about  the  same 
chance  of  passing  (96%),  but  fewer  Soldiers  would  be  eligible  (42%). 

Figure  1  shows  the  relationship  between  MM  and  the  probability  of  passing  for 
high  school  educated  male  Soldiers,  including  upper  and  lower  bounds  based  upon  the 
standard  error  of  the  estimated  MM  coefficient.  For  a  particular  MM  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
corresponding  probability  of  passing.  For  example,  a  MM  value  of  100  corresponds  to  a 
passing  probability  of  about  93%  for  a  high  school  educated  male  Soldier. 
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Appendix  A:  Soldier  Characteristics  (67T) 


Gender 

Outcome 

Male 

Female 

fail 

3.5% 

6.1% 

pass 

96.5% 

93.9% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

0.0% 

4.0% 

2.1% 

pass 

100.0% 

96.0% 

97.9% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

4.3% 

5.4% 

1.7% 

pass 

95.7% 

94.6% 

98.3% 

77 


15U  (67U):  CH-47  Helicopter  Repairer 

The  final  sample  included  972  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (91.5%)  or  failed  for  academic  reasons  (8.5%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (97%,  3%  female),  most  had  a  high  school 
diploma  but  not  more  education  (88%,  7%  some  college,  5%  GED  or  less),  and  most 
were  from  the  Regular  Army  (69%,  21%  Guard,  10%  Reserve).  The  governing  AA 
composite,  Mechanical  Maintenance  (MM),  for  this  MOS  has  a  cutoff  score  of  102;  the 
sample  mean  is  115.11  (standard  deviation  =  10. 171).  See  the  Appendix  for  a  description 
of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  model,  including  the  MM  composite,  education  status  (GED 
=  GED  or  less  than  a  high  school  diploma,  COLLEGE  =  some  college  education),  Anny 
component  (GUARD  =  National  Guard,  RESERVE  =  Army  Reserve),  and  gender 
(GENDER  =  female),  accounts  for  about  four  percent  of  the  variation  in  the  dependent 

variable.  This  model  is  statistically  significant  (x  =  17.649,  P  =  .007),  and  has  limited 

2 

explanatory  power  (Nagelkerke  R“  =  .041). 
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Table  1.  67U:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

17.649** 

Log  Likelihood 

549.498 

Nagelkerke  R  Square 

.041 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-.850 

.427 

.581 

GED 

-4.53 

.636 

.633 

COLLEGE 

-.125 

.883 

.076 

GUARD 

.867 

2.379 

5.571* 

RESERVE 

.998 

2.712 

3.364 

GENDER 

-.362 

.696 

.420 

MM 

.027 

1.027 

7.467** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 


There  is  a  statistically  significant  effect  for  the  MM  composite.  The  model 
suggests  that  having  a  higher  MM  composite  score  increases  the  odds  of  passing.  At  the 
mean  MM  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  three 
percent  in  the  odds  of  passing  the  course,  and  a  five-point  increase  in  MM  would  increase 
the  odds  of  passing  by  14%.  There  is  also  a  significant  effect  for  GUARD  where 
guardsmen  have  138%  increased  odds  of  passing. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  probabilities  were  calculated  using  the  formula  for  finding  probability  in  a 
binary  logistic  model,  which  is 

Q(P'X>  /  (1  +  Q(fl’x)y 

Because  GED,  COLLEGE,  RESERVE,  and  GENDER  were  not  statistically  significant, 
this  analysis  is  confined  to  the  modal  demographic — Regular  Army  high  school  educated 
male  Soldiers  (though  results  would  be  similar  for  any  demographic  combination,  except 
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National  Guardsmen).  Based  on  the  model,  when  the  cutoff  score  is  at  its  current  levels 
(MM  =  102),  male  Soldiers  with  a  high  school  diploma  and  an  average  MM  score  (MM  = 
1 13.76)  have  approximately  a  90%  chance  of  passing.  Currently,  about  55%  of  Soldiers 
are  eligible  for  MOS  67U  assignment  at  the  current  cutoff.  Lowering  the  MM  cutoff  by 
five  points  (MM  =  97)  would  increase  eligibility  by  13  percentage  points,  while  the 


Table  2.  67U:  Probability  that  a  Soldier  (from  the  larger  Army  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model 

Cutoff: 
MM  =  92 

Cutoff: 
MM  =  97 

Cutoff: 
MM  =  102 

Cutoff: 
MM  =  107 

Cutoff: 
MM  =  112 

Percent  Eligible 
(Regular  Army) 

78.8% 

67.6% 

55.0% 

42.0% 

29.3%> 

Mean:  MM 

108.60 

111.03 

113.76 

116.80 

120.09 

Passing  rates: 

High  School  Male 

88.9% 

89.5% 

90.2% 

90.9% 

91.6% 

average  male  Soldier  who  would  qualify  for  training  would  have  essentially  the  same 
chance  of  passing  (90%).  Raising  the  MM  cutoff  scores  by  five  points  (MM  =  107),  the 
average  male  Soldier  who  would  still  qualify  for  the  MOS  would  have  about  the  same 
chance  of  passing  (91%),  but  fewer  Soldiers  would  be  eligible  (42%). 

Figure  1  shows  the  relationship  between  MM  and  the  probability  of  passing  for 
high  school  educated  male  Soldiers,  including  upper  and  lower  bounds  based  upon  the 
standard  error  of  the  estimated  MM  coefficient.  For  a  particular  MM  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
corresponding  probability  of  passing.  For  example,  a  MM  value  of  100  corresponds  to  a 
passing  probability  of  about  86%  for  a  high  school  educated  male  Soldier. 
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Chance  of  Passing 


Figure  1:  Predicted  Probability  of  Passing:  High  School  Male  Soldiers 


^^^“Male  -  -  -  |\y|a|e  Upper  -  -  -  Male  Lower 
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Appendix  A:  Soldier  Characteristics  (67U) 


Gender 

Outcome 

Male 

Female 

fail 

8.4% 

mgsm 

pass 

91.6% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

9.1% 

8.5% 

8.3% 

pass 

90.9% 

91.5% 

91.7% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

10.2% 

4.3% 

4.9% 

pass 

89.8% 

95.7% 

95.1% 
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21B:  Combat  Engineer 


The  final  sample  included  1902  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (96.4%)  or  failed  for  academic  reasons  (3.6%)  were 
included  in  the  analysis  sample  of  1902  Soldiers  because  we  assume  that  non-completion 
for  non-academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  all  of  the  Soldiers  were  male  as  this  MOS  is  not  open  to  women, 
most  had  a  high  school  diploma  but  not  more  education  (79%,  1 1%  some  college,  10% 
GED  or  less),  and  the  greatest  number  were  from  Regular  Army  (54%,  41%  National 
Guard,  5%  Army  Reserve).  The  governing  AA  composite,  Combat  (CO),  for  this  MOS 
has  a  cutoff  score  of  87;  the  sample  mean  is  105.99  (standard  deviation  =  13.201).  See 
the  Appendix  for  a  description  of  Soldier  characteristics  for  this  MOS  training  sample. 


Table  1.  2  IB:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

28.484*** 

Log  Likelihood 

558.092 

Nagelkerke  R  Square 

.056 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-1.057 

.348 

1.452 

GUARD 

.773 

2.167 

7.143** 

CO 

.040 

1.041 

21.394*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 


A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  a  model  including  the  CO  composite,  as  well  as  Army 
component  (GUARD  =  National  Guard),  accounts  for  almost  six  percent  of  the  variation 
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in  the  dependent  variable.  This  model  is  statistically  significant  fy2  =  28.484,  P  <  .001), 
but  its  explanatory  power  is  limited  (Nagelkerke  R~  =  .056). 

There  is  a  statistically  significant  effect  for  the  CO  composite.  The  model 
suggests  that  having  a  higher  CO  composite  score  increases  the  odds  of  passing.  At  the 
mean  CO  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  4%  in  the 
odds  of  passing  the  course,  and  a  five-point  increase  in  CO  would  increase  the  odds  of 
passing  by  22%.  There  is  also  a  significant  effect  for  Guard  where  members  of  the 
National  Guard  have  1 17%  greater  odds  of  passing  the  course. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

q(P’x)  /  (1  +  q(P’x>). 

This  analysis  is  confined  to  Soldiers  from  the  Regular  Army.  Based  on  the  model,  when 
the  cutoff  score  is  at  its  current  level  (CO  =  87),  regular  Anny  Soldiers  with  an  average 
CO  score  (CO  =  107.20)  have  approximately  a  96%  chance  of  passing.  Currently,  about 
93%  of  Soldiers  are  eligible  for  MOS  2  IB  assignment  at  the  current  cutoff.  Lowering  the 
cutoff  by  five  points  (CO  =  82)  would  increase  eligibility  by  two  percentage  points  (to 
98%)  and  the  average  Soldier  who  would  qualify  for  training  would  practically  have  the 
same  chance  of  passing  (96%).  Raising  the  cutoff  score  by  five  points  (CO  =  92),  the 
average  Soldier  would  have  the  same  chance  of  passing  (96%)  but  eligibility  would  fall 
by  about  nine  percentage  points. 
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Table  2.  2  IB:  Probability  that  a  male  Regular  Army  Soldier  (from  the  larg 
contract  population)  will  pass  the  course  based  on  the  binary  logistic  mode 

er  Army 

Cutoff  = 

83 

Cutoff = 

88 

Cutoff = 

93 

Cutoff = 

98 

Cutoffs 

113 

Percent  Eligible 
(Regular  Army) 

99.4% 

97.6% 

92.8% 

84.2% 

72. 7% 

Mean 

105.61 

106.08 

107.02 

109.04 

111.04 

Passing  rate: 

High  school  male 

96.0% 

96.0% 

96.2% 

96.5% 

96.7% 

Figure  1  shows  the  relationship  between  CO  and  the  probability  of  passing  for 
Regular  Army  Soldiers,  including  upper  and  lower  bounds  based  upon  the  standard  error 
of  the  estimated  CO  coefficient.  For  a  particular  CO  score,  trace  a  vertical  line  up  to  the 
curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the  corresponding 
probability  of  passing.  For  example,  a  CO  score  of  100  corresponds  to  a  passing 
probability  of  about  95%  for  a  Regular  Army  male  Soldier.  One  can  see  that  Soldiers 
with  a  relatively  low  CO  score  have  a  reasonable  chance  of  passing  the  course.  As  CO 
scores  increase,  the  chance  that  a  Soldier  will  pass  the  course  increases. 
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Appendix:  Soldier  Characteristics  (2 IB) 


Gender 

Male 

Female 

fail 

3.6% 

.0% 

pass 

96.4% 

.0% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

.0% 

4.4% 

pass 

100.0% 

95.6% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

.0% 

pass 

100.0% 
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25B  (74B):  Information  Systems  Operator-Analyst 


The  final  sample  included  2360  soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  soldiers  who  graduated  (92.7%)  or  failed  for  academic  reasons  (7.3%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (74%,  26%  female),  most  had  a  high  school 
diploma  but  not  more  education  (80%,  17%  some  college,  3%  GED  or  less),  and  most 
were  Regular  Army  (72%,  10%  Army  Reserve,  18%  National  Guard).  The  governing  AA 
composite,  Skilled  Technical  (ST),  for  this  MOS  has  a  cutoff  score  of  96;  the  sample 
mean  is  1 10.55  (standard  deviation  =  10.905).  See  the  Appendix  for  a  description  of 
Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 


Table  1.  74B:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

57.942*** 

Log  Likelihood 

1179.196 

Nagelkerke  R  Square 

.059 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-2.803 

.061 

8.241** 

GED 

.434 

1.543 

.501 

COLLEGE 

.575 

1.777 

4.180* 

RESERVE 

.744 

2.104 

4.371* 

GUARD 

.472 

1.603 

3.619 

GENDER 

.265 

1.304 

1.985 

ST 

.047 

1.048 

26.964*** 

*  =  p  <  .05  ***  =  p<.001 

**  =  p  <  .005 
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Table  1  and  indicate  that  a  model  including  the  ST  composite,  education  status  (GED  = 
GED  or  less  than  high  school  diploma;  COLLEGE  =  some  college),  Army  component 
(GUARD  =  National  Guard;  RESERVE  =  Army  Reserve),  and  gender  (GENDER  = 
female)  accounts  for  only  about  six  percent  of  the  variation  in  the  dependent  variable. 

This  model  is  statistically  significant  (x2  =  57.942,  P  <  .001),  but  its  explanatory  power  is 
somewhat  limited  (Nagelkerke  R2  =  .059). 

There  are  statistically  significant  effects  for  the  ST  composite,  having  some 
college  education,  and  Reserve  membership.  The  model  suggests  that  having  a  higher  ST 
composite  score  increases  the  odds  of  passing.  At  the  mean  ST  score,  an  increase  of  one 
point  is  associated  with  an  increase  of  about  five  percent  in  the  odds  of  passing  the 
course,  and  a  five-point  increase  in  ST  would  increase  the  odds  of  passing  by  26%.  There 
is  also  a  noteworthy  effect  for  COLLGE:  having  a  college  education  increases  the  odds  of 
passing  by  78%.  Finally,  there  is  a  noteworthy  effect  for  RESERVE,  where  the  odds  of  a 
Reservist  passing  this  training  exceed  that  of  the  average  Regular  Anny  Soldier  by  1 10%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

Q(f)’x)  /  (1  +  Q(P’x)y 

Because  GED,  GENDER,  and  GUARD  were  not  statistically  significant,  this  analysis  is 
confined  to  the  modal  demographics  factored  by  COLLEGE:  Soldiers  from  the  Regular 
Army  with  a  high  school  or  college  education  (though  results  would  be  similar  for  any 
demographic  combination,  except  Reservists).  Based  on  the  model,  when  the  cutoff  score 
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is  at  its  current  level  (ST  =  96),  male  soldiers  with  a  high  school  diploma  and  an  average 
ST  score  (ST  =  1 13.07)  have  approximately  a  92%  chance  of  passing,  while  their 
counterparts  with  some  college  have  approximately  a  95%  chance  of  passing.  Currently, 
about  81%  of  Soldiers  are  eligible  for  MOS  74B  assignment  at  the  current  cutoff  (ST  = 
96).  Lowering  the  cutoff  by  five  points  (ST  =  91),  eligibility  would  increase  by  10 
percentage  points  (to  91%)  while  the  average  Soldier  who  would  qualify  for  training 
would  have  a  about  the  same  chance  of  passing  (high  school  =  91%;  college  =  95%). 
Raising  the  cutoff  score  by  five  points  (ST  =  101),  the  average  male  Soldier  with  either  a 
high  school  diploma  or  some  college  education  would  have  a  slightly  higher  chance  of 
passing  (high  school  =  93%;  college  =  96%)  but  eligibility  would  fall  by  about  13 
percentage  points  (68%). 


Table  2.  74B:  Probability  that  a  Soldier  (from  the  larger  Anny  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model 

Cutoff  = 

86 

Cutoff = 

91 

Cutoff = 

96 

Cutoff = 
101 

Cutoff = 
106 

Percent  Eligible 
(Regular  Army) 

98.1% 

91.0% 

80.5% 

67.8% 

52.7% 

Mean 

107.41 

109.06 

111.14 

113.60 

116.55 

Passing  rates: 

High  School  &  GED 
male 

90.4% 

91.1% 

91.8% 

92.7% 

93.6% 

College  male 

94.4% 

94.8% 

95.2% 

95.7% 

96.2% 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  on  the 
first  attempt  for  Regular  Army  Soldiers  factored  by  attending  some  college,  including 
upper  and  lower  bounds  based  upon  the  standard  error  of  the  estimated  coefficient.  For  a 
particular  ST  score,  trace  a  vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to 
the  axis  to  determine  the  corresponding  probability  of  passing.  For  example,  an  ST  score 
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of  100  corresponds  to  a  passing  probability  of  about  87%  for  a  high  school  educated, 
Regular  Anny  male  Soldier.  One  can  see  that  while  there  is  a  steady  increase  in  the 
probability  of  passing,  both  high  school  and  college  Soldiers  reach  a  high  probability  of 
passing  at  relatively  low  ST  scores. 


Figure  1:  Predicted  Probalility  of  Passing:  Male  Soldiers 


—  High  School  -  -  -  HS  Upper  -  -  -  HS  Lower  College  —  -  Col  Upper  —  -  Col  Lower 


Testing  a  Central  Assumption  of  the  Approach 

Central  to  our  approach  has  been  the  estimation  of  model  parameters  using  only 
academic  passes  and  failure  cases  -  we  specifically  exclude  failure  cases  that  have  been 
identified  with  non-academic  reasons  for  failure.  In  doing  so  we  assume  that  non¬ 
completion  for  non-academic  reasons  is  not  related  to  cognitive  criteria.  In  this  section 
we  report  on  our  attempt  examine  the  reasonableness  of  this  assumption. 

When  we  include  non-academic  failure  cases  in  the  data  sample,  we  would  expect 
the  goodness-of-fit  to  deteriorate  and  to  see  smaller  predicted  composite  effects,  if  higher 
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scoring  Soldiers  predominate  in  these  “additional”  cases.  Alternatively,  if  lower  scoring 
Soldiers  predominate  in  these  cases,  it  is  possible  to  see  little  or  no  deterioration  in  the  fit 
of  the  model.  In  Table  4  we  present  the  regression  model  results  when  an  additional  460 
non-academic  failure  cases  are  included  in  the  analysis  sample  (for  a  total  of  2820  cases). 
There  is  deterioration  in  the  explanatory  power  of  the  model:  the  Nagelkerke  R2  declines 
from  0.059  in  the  original  sample  to  0.049  in  the  augmented  sample.  In  addition,  the  ST 
composite  effect  falls  from  1 .048  to  1 .023  in  the  augmented  sample. 


Table  4.  74B:  Results  of  the  binary  logistic 
prediction  model  including  academic  failure  and 
non-academic  failure  cases 


Chi-  Square 

91.825*** 

Log  Likelihood 

2911.504 

Nagelkerke  R  Square 

.049 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-1.380 

.251 

7.296** 

GED 

-.219 

.803 

.603 

COLLEGE 

.249 

1.282 

3.337 

GUARD 

.747 

2.110 

24.922*** 

RESERVE 

.785 

2.192 

16.847*** 

GENDER 

-.127 

.881 

1.437 

ST 

.023 

1.023 

23  974*** 

*  =  p  <  .05  **  =  p  <  .01 

***  =  p  <  .001 


An  even  more  extreme  comparison  might  be  between  the  original  sample  and  a 
sample  utilizing  only  the  non-academic  failure  cases  -  i.e.,  replacing  the  academic  failure 
cases  in  the  original  sample  with  non-academic  failure  cases.  The  hypothesis  is  that  the 
latter  bring  increased  randomness  to  the  data  set,  and  that  the  resulting  regression  model 
will  not  fit  as  well,  and  the  estimated  governing  composite  effects  will  be  weaker.  In 
Table  5  we  present  the  regression  model  results  when  only  non-academic  failure  cases 
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are  included  (along  with  academic  pass  cases).  This  data  sample  has  a  total  of  2647 
observations.  We  find  even  greater  deterioration  in  the  explanatory  power  of  the  model: 
the  Nagelkerke  R"  in  the  “non-academic  failure”  model  is  about  65%  that  of  the  original 
sample  model  (.039  versus  .059).  In  addition,  ST  composite  effect  is  considerably 
weaker:  1.013  for  the  sample  including  non-academic  failures  compared  to  1.047  for  the 
original  sample. 


Table  5.  74B:  Results  of  the  binary  logistic 
prediction  model  including  only  non-academic 
failure  cases 


Chi-  Square 

63.518*** 

Log  Likelihood 

2381.421 

Nagelkerke  R  Square 

.039 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

.063 

1.065 

.013 

GED 

-.409 

.665 

1.783 

COLLEGE 

.162 

1.176 

1.190 

GUARD 

.841 

2.318 

22.683*** 

RESERVE 

.790 

2.204 

13.142*** 

GENDER 

-.286 

.752 

5.875* 

ST 

.013 

1.013 

6.882** 

*  =  p<  .05  **  =  p<  .01 

***  =  p<  .001 


The  results  of  these  additional  analyses  support  the  approach  taken  to  include  only 
academic  pass  and  failure  cases  in  the  data  analysis  sample.  The  non-academic  failure 
cases  were  found  to  vary  in  a  less  predictable  way  with  cognitive  criteria  and,  we 
conclude,  to  represent  the  outcome  of  a  different  process. 
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Appendix:  Soldier  Characteristics  (74B) 


Gender 

Male 

Female 

Outcome 

fail 

7.1% 

8.1% 

pass 

92.9% 

91.9% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

pass 

3.8% 

96.2% 

8.2% 

91 .8% 

3.8% 

96.2% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

pass 

8.4% 

91.6% 

3.7% 

96.3% 

5.1% 

94.9% 
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74B  Risk  Analysis  of  Policy  Alternatives 


In  this  section  we  extend  the  policy  analysis  with  a  risk  analysis  of  the  policy 
alternatives  considered  using  simulation  methods.  We  employ  the  “@RISK”  software 
package  which  is  designed  as  an  add-in  for  Microsoft  Excel.14  There  are  four  steps  in  the 
risk  analysis. 

The  first  step  is  developing  a  model.  A  regression  model  for  predicting  the 
passing  rate  has  been  specified  and  estimated,  and  the  results  stored  in  Excel  worksheet 
format.  See  Table  1. 

The  second  step  is  identifying  the  uncertainty  inherent  in  both  the  input  variables 
(composite  scores,  Soldier  characteristics)  and  the  corresponding  estimated  parameters 
and  specifying  their  possible  values  with  probability  distributions. 

To  approximate  the  governing  composite  distribution,  we  use  the  larger  Army 
contract  population  (EAF  file:  those  who  contracted  from  Jan  1992  to  Aug  2003).  This 
distribution  can  be  assumed  to  follow  a  normal  distribution.  The  mean  and  standard 
deviation  estimated  from  the  contract  population  and  used  as  input  to  the  simulations  are 
shown  in  Table  R-l  (before  truncation). 


Table  R-l.  Input  infonnation  for  the  simulations 

Mean 

Std  error  / 
deviation 

Constant 

-2.803 

.976 

ST  var 

107.02  (before  truncation) 

12.157 

ST  coeff 

.047 

.009 

GED  var 

3.3% 

GED  coeff 

.434 

.613 

COLLEGE  var 

16.9% 

COLLEGE  coeff 

.575 

.281 

GUARD  var 

17.5% 

GUARD  coeff 

.472 

.248 

RESERVE  var 

10.3% 

RESERVE  coeff 

.744 

.356 

GENDER  var 

25.6% 

GENDER  coeff 

.265 

.188 

14  Software  is  available  from  Palisade  Corporation,  Newfield,  NY. 
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To  approximate  the  distributions  for  the  demographic  variables  we  rely  on  the 
training  data  sample  used  to  estimate  the  regression  model.  These  distributions  can  be 
assumed  to  follow  binomial  distributions  with  means  estimated  from  the  training  data 
sample  as  shown  in  Table  R-l. 

To  approximate  the  distributions  for  the  estimated  parameters  in  the  logit  model, 
we  note  that,  under  very  general  conditions,  the  estimates  will  be  asymptotically  normally 
distributed.15  The  estimated  coefficients  (means  and  standard  errors)  are  shown  in  Table 
R-l. 


The  third  step  is  analyzing  the  model  with  simulation  to  determine  the  range  and 
probabilities  of  all  possible  training  outcome  results.  The  results  reported  represent  the 
simulation  of  the  regression  prediction  equation  for  1000  iterations  using  a  fixed  seed.16 
The  comparisons  are  between  the  baseline  case  (ST  =  96)  and  the  policy  cases  (ST  =  91 
and  ST  =  101).  These  cases  are  delineated  by  the  cutoff  score  level  which  serves  as  the 
lower  truncation  point  in  the  governing  composite  input  distribution.  The  simulation 
results  in  Table  R-2  describe  the  effect  on  the  mean  probability  of  passing  as  the  ST 
cutoff  score  level  is  varied,  and  represent  an  improvement  of  the  policy  analysis 
presented  in  Table  2  because  they  reflect  the  simultaneous  effects  of  all  known  sources  of 
uncertainty. 


Table  R-2.  74B  simulation  resul 

Its 

on 

H 

II 

kO 

ST  =  96 

ST  =  101 

Mean  governing 
composite 

109.27 

110.95 

113.24 

Mean  probability 
of  passing 

87.52% 

88.17% 

89.16% 

Std  deviation 

15.08% 

14.14% 

13.20% 

The  frequency  distributions  of  simulated  effects  are  shown  below  in  the  histogram 
graphs.  The  histogram  portrays  the  full  range  of  outcomes  that  average  to  the  mean 
effect  as  reported.  Looking  at  the  histogram  for  the  baseline  case  (ST  =  96),  probability 
of  passing  is  plotted  on  the  horizontal  axis  and  the  height  of  each  bar  indicates  the 
relative  number  of  Soldiers  at  that  passing  probability.  The  information  in  the  histograms 
is  effectively  summarized  in  Table  R-3.  In  that  table  we  see  the  percentage  of  Soldiers 


15  The  intuition  is  that  logit  estimates  are  derived  from  maximum  likelihood  (ML)  estimation,  and  under 
fairly  general  conditions  ML  estimates  are  consistent  and  asymptotically  normally  distributed. 

16  Latin  Hypercube  (LH)  sampling  is  utilized  in  drawing  samples  for  the  input  distributions.  It  is  designed 
to  accurately  recreate  the  input  distribution  through  sampling  in  fewer  iterations  when  compared  with  the 
Monte  Carlo  method.  The  key  to  LH  sampling  is  stratification  of  the  input  probability  distributions. 
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who  meet  or  exceed  a  given  chance  of  passing.  For  example,  looking  at  the  baseline  case 
(ST  =  96),  about  89%  of  Soldiers  have  a  70%  or  greater  chance  of  passing. 


Table  R-3. 

Percent  of  simulated  Soldiers  that  meet  or  exceed 
given  chance  of  passing 


Chance  of 
passing 

ST  =  91 

ST  =  96 

ST  =  101 

95.0% 

41.2% 

43.7% 

46.5% 

90.0% 

62.3% 

63.1% 

66.3% 

85.0% 

72.1% 

73.6% 

76.9% 

80.0% 

79.7% 

79.1% 

82.5% 

75.0% 

84.8% 

85.8% 

86.6% 

70.0% 

89.0% 

88.7% 

89.1% 

65.0% 

91.0% 

91.6% 

93.3% 

60.0% 

93.0% 

94.0% 

95.1% 

55.0% 

94.8% 

95.6% 

96.3% 

50.0% 

96.0% 

96.9% 

97.6% 

We  find  that  the  mean  probability  of  passing  in  the  simulation  is  88%  (Table  R-2, 
baseline).  Looking  at  the  histograms,  we  see  the  largest  portion  of  Soldiers  is  in  the 
highest  range  of  passing  scores,  trending  sharply  downward  as  the  probability  of  passing 
approaches  zero.  We  note  that  the  mean  simulated  passing  probability  is  somewhat  lower 
but  comparable  to  the  static  prediction  at  the  mean  ST  score  (Table  2).  In  the  simulated 
predictions  we  are  calculating  an  average  over  all  Soldiers  and  taking  into  account  all 
sources  of  uncertainty,  whereas  in  the  static  prediction  we  are  calculating  the  probability 
of  passing  only  at  the  mean  ST  score,  ignoring  the  distribution  of  Soldier  scores  etc.  as 
well  as  uncertainty.  While  recognizing  this  variability  is  important,  we  also  note  that  the 
policy  case  impacts  relative  to  the  baseline  are  the  same  in  both  simulation  and  static 
prediction  modes. 

The  fourth  step  is  making  a  decision  based  on  the  results  provided  and  the 
preferences  of  the  school  proponent.  This  involves  the  evaluation  by  the  school 
proponent  of  the  tradeoff  between  increased  eligibility  and  reduced  passing  probabilities 
(and  vice-versa). 
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98 


Distribution  for  PASS  EQUATION  91 


X<=0.55  X  <=0.99 
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25D  (formerly  74C):  Telecommunications  Operator-Maintainer 


The  final  sample  included  739  soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  soldiers  who  graduated  (97%)  or  failed  for  academic  reasons  (3%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (66%,  34%  female),  most  had  a  high  school 
diploma  but  not  more  education  (89%,  6%  some  college,  5%  GED  or  less),  and  most 
were  Regular  Army  (69%,  15%  Army  Reserve,  15%  National  Guard).  The  governing  AA 
composites,  Electrical  Repair  (EL)  and  Surveillance  /  Communication  (SC),  for  this  MOS 
have  cutoff  scores  of  89  and  90,  respectively;  the  sample  mean  of  EL  is  101.87  (standard 
deviation  =  1 1.339),  and  the  sample  mean  of  SC  is  100.63  (standard  deviation  =  10.670). 


Table  1.  25D:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

8.509 

Log  Likelihood 

189.457 

Nagelkerke  R  Square 

.04 

9 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

.740 

2.096 

.207 

GED 

-.404 

.668 

.136 

COLLEGE 

-1.004 

.366 

2.340 

RESERVE 

-.235 

.790 

.184 

GUARD 

1.325 

3.763 

1.553 

GENDER 

.661 

1.937 

1.591 

EL 

-.003 

.997 

.004 

SC 

.029 

1.029 

.477 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .005 
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See  the  Appendix  for  a  description  of  Soldier  characteristics  for  this  MOS  training 
sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  a  model  including  the  EL  and  SC  composites,  education  status 
(GED  =  GED  or  less  than  high  school  diploma;  COLLEGE  =  some  college),  Army 
component  (GUARD  =  National  Guard;  RESERVE  =  Army  Reserve),  and  gender 
(GENDER  =  female)  accounts  for  only  about  five  percent  of  the  variation  in  the 
dependent  variable.  This  model  is  not  statistically  significant  (y2  =  8.509,  P  =  .290),  with 
limited  explanatory  power  (Nagelkerke  R“  =  .049). 

There  are  no  statistically  significant  effects  for  any  of  the  variables  in  the 
equation.  The  model  suggests  that  having  a  higher  EL  or  SC  composite  score  has  no 
consistent  effect  upon  the  odds  of  passing,  but  there  are  probably  too  few  failure  cases  for 
definitive  conclusions. 

Policy  Analysis.  Because  the  model  estimated  for  25D  (74C)  is  not  significant 
and  has  little  explanatory  power,  no  policy  analysis  can  be  developed  from  the  existing 
data. 
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Appendix:  Soldier  Characteristics  (25D) 


Gender 

Outcome 

Male 

Female  | 

fail 

1 1 

pass 

98.0% 

Education  Level  I 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

2.7% 

6.5% 

pass 

97.0% 

97.3% 

93.5% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

.9% 

4.4% 

pass 

96.9% 

99.1% 

95.6% 
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25M:  Media  Illustrator 


The  final  sample  included  258  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  soldiers  who  graduated  (96.5%)  or  failed  for  academic  reasons  (3.5%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (67%,  33%  female),  most  had  a  high  school 
diploma  or  less  education  (78%,  17%  some  college,  5%  GED  or  less),  and  the  greatest 
number  were  Regular  Army  (48%,  28%  Army  Reserve,  24%  National  Guard).  The 
governing  AA  composites,  Electrical  Repair  (EL)  and  Skilled  Technical  (ST),  for  this 
MOS  have  cutoff  scores  of  93  and  92,  respectively;  the  EL  sample  mean  is  107.60 
(standard  deviation  =  12.712)  and  the  ST  sample  mean  is  109.01  (standard  deviation  = 

1 12.775).  See  the  Appendix  for  a  description  of  Soldier  characteristics  for  this  MOS 
training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcome  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  a  model,  including  the  EL  and  ST  composites,  accounts  for 


Table  1.  25M:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

0.614 

Log  Likelihood 

77.471 

Nagelkerke  R  Square 

.009 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

.1.575 

4.832 

.541 

ST 

-.018 

.983 

EL 

.034 

1.035 

*  =  n  <  .05  ***  =  n<.001 


103 


about  0.9  percent  of  the  variation  in  the  dependent  variable.  This  model  is  not  statistically 
significant  (x  =  0.614,  P  =  .736),  with  little  explanatory  power  (Nagelkerke  R“  =  .009). 

There  are  no  statistically  significant  effects  for  any  of  the  variables  in  the 
equation.  The  data  are  insufficient  (too  few  failure  observations)  to  determine  if  any  of 
the  variables,  including  EL  or  ST,  have  a  consistent  effect  on  passing. 

Policy  Analysis.  Because  the  model  estimated  for  25M  is  not  significant  and  has 
little  explanatory  power,  no  policy  analysis  can  be  developed  from  the  existing  data. 
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Appendix:  Soldier  Characteristics  (25M) 


Gender 

Outcome 

Male 

Female  | 

fail 

4.1% 

2.3% 

pass 

95.9% 

97.7% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

.0% 

4.0% 

2.2% 

pass 

100.0% 

96.0% 

97.8% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

2.4% 

6.9% 

97.6% 

93.1% 
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25R:  Visual  Information  Equipment  Operator 


The  final  sample  included  105  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  soldiers  who  graduated  (86.7%)  or  failed  for  academic  reasons  (13.3%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (81%,  19%  female),  most  had  a  high  school 
diploma  or  less  education  (81%,  18%  some  college,  1%  GED  or  less),  and  the  greatest 
number  were  Regular  Army  (70%,  30%  Army  Reserve).  The  governing  AA  composite, 
Electrical  Repair  (EL),  for  this  MOS  has  a  cutoff  score  of  107;  the  sample  mean  is  1 17.30 
(standard  deviation  =  7. 186).  See  the  Appendix  for  a  description  of  Soldier 
characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcome  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  a  model  including  the  EL  composite,  Army  component 


Table  1.  25R:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

1.565 

Log  Likelihood 

80.897 

Nagelkerke  R  Square 

.027 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

1.977 

7.222 

.153 

RESERVE 

-.688 

.503 

1.306 

GENDER 

.405 

1.499 

.222 

EL 

.001 

1.001 

.000 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .005 
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(RESERVE  =  Army  Reserve),  and  gender  (GENDER  =  female)  accounts  for  about  three 
percent  of  the  variation  in  the  dependent  variable.  This  model  is  not  statistically 
significant  (%  =  1.565,  P  =  .667),  and  its  explanatory  power  is  limited  (Nagelkerke  R“  = 
.027). 

There  are  no  statistically  significant  effects  for  any  of  the  variables  in  the 
equation.  The  data  are  insufficient  (too  few  failure  observations)  to  determine  if  any  of 
the  variables,  including  EL,  have  a  consistent  effect  upon  passing. 

Policy  Analysis.  Because  the  model  estimated  for  25R  is  not  significant  and  has 
little  explanatory  power,  no  policy  analysis  can  be  developed  from  the  existing  data. 
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Appendix:  Soldier  Characteristics  (25R) 


Gender  1 

Outcome 

Male 

Female  [ 

fail 

14.1% 

10.0% 

pass 

85.9% 

90.0% 

Education  Level  [ 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

.0% 

16.5% 

.0% 

pass 

100.0% 

83.5% 

100.0% 

Component  j 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

10.8% 

19.4% 

.0% 

pass 

89.2% 

80.6% 

.0% 
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25V:  Combat  Documentation/Production 


The  final  sample  included  215  soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (93%)  or  failed  for  academic  reasons  (7%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  this  sample,  most  Soldiers  were  male  (63%,  37%  female),  most  had  a  high 
school  diploma  but  not  more  education  (82%,  17%  some  college,  1%  GED  or  less),  and 
most  were  Regular  Army  (70%,  24%  Army  Reserve,  6%  National  Guard).  The  governing 
AA  composites,  Skilled  Technical  (ST)  and  Electronics  Repair  (EL),  for  this  MOS  have 
cutoff  scores  of  92  and  93,  respectively;  the  sample  mean  for  ST  is  109.00  (standard 
deviation  =  15.03),  and  for  EL  it  is  107.19  (standard  deviation  =  14.92).  See  the 
Appendix  for  a  description  of  Soldier  characteristics  for  this  MOS  training  sample. 


Table  1.  25V:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

8.509* 

Log  Likelihood 

100.297 

Nagelkerke  R  Square 

.098 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

.666 

1.947 

.237 

GENDER 

-1.305 

.271 

5.061* 

ST 

.025 

1.025 

3.578x 

x  =  p  <  .06 

*  =  p  <  .05 


A  forward  stepwise  binary  logistic  regression  model  was  estimated  to  explain 
pass/fail  AIT  outcomes  and  focused  on  the  first  training  attempt.  This  type  of  model 
inserts  (or  removes)  variables,  one  at  a  time,  in  order  to  maximize  its  predictive  power. 
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The  estimation  results  are  reported  in  Table  1  and  indicate  that  a  model  including  the  ST 
composite17  and  gender  (GENDER  =  female)  accounts  for  about  ten  percent  of  the 
variation  in  the  dependent  variable.  This  model  is  statistically  significant  (%2  =  8.059,  P  = 
.014),  with  moderate  explanatory  power  (Nagelkerke  R“  =  .098). 

The  effect  of  ST  is  not  statistically  significant  at  the  standard  .05  level,  though  it 
approaches  significance  (p  =  .059).  With  this  caveat,  an  increase  of  one  point  in  ST  is 
associated  with  an  increase  of  almost  three  percentage  points  in  the  odds  of  passing  the 
course,  and  a  five-point  increase  in  ST  would  increase  the  odds  of  passing  by  33%. 

There  are  statistically  significant  effects  for  GENDER.  The  model  suggests  that  females 
are  less  likely  to  pass  -  the  odds  of  a  female  Soldier  passing  are  73%  lower  than  for 
males. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Army  contract  population  would  pass  the  course  on  the  first  attempt  based  upon 
the  binary  logistic  model.  These  were  calculated  using  the  formula  for  finding  probability 
in  a  binary  logistic  model,  which  is 

Q(f>’x>  /  (1  +  Q(V*)y 

Based  on  the  model,  when  the  cutoff  score  is  at  its  current  level  (ST  =  92  and  EL  =  93), 
male  Soldiers  with  average  ST  scores  (ST  =  1 10.09)  have  approximately  a  97%  chance  of 
passing,  while  their  female  counterparts  have  approximately  an  89%  chance  of  passing. 
Currently,  about  89%  of  soldiers  qualify  for  MOS  25  V  assignment  at  the  current  cutoff 
(ST  =  92  &  EL  =  93).  Lowering  the  cutoff  by  five  points  (ST  =  87  &  EL  =  88)  increases 
eligibility  by  eight  percentage  points,  and  the  average  Soldier  who  would  qualify  for 

17  EL  was  excluded  during  the  stepwise  regression  procedure  as  it  was  strongly  correlated  with  ST,  which 
was  a  better  predictor  of  passing/failing. 
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training  would  have  essentially  the  same  chance  of  passing  (male  =  97%;  female  =  89%). 
Raising  the  cutoff  score  by  five  points  (ST  =  97  &  EL  =  98),  the  average  Soldier  who 
would  still  qualify  for  the  MOS  would  have  the  essentially  same  chance  of  passing  (male 


Table  2.  25V:  Probability  that  a  Soldier  (from  the  larger  Army  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model  _ _ 


Cutoff  = 

Cutoff = 

Cutoff = 

Cutoff = 

Cutoff = 

82/83 

87/88 

92/93 

97/98 

102/103 

Percent  Eligible 
(Regular  Army) 

99.8% 

97.1% 

89.0% 

78.2% 

64.9% 

Mean:  ST 

107.01 

108.18 

110.09 

112.38 

115.07 

Passing  rates: 

Male 

96.6% 

96.7% 

96.8% 

97.0% 

97.2% 

Female 

88.5% 

88.8% 

89.2% 

89.8% 

90.4% 

=  97%;  female  =  90%),  but  fewer  Soldiers  would  be  eligible  (78%). 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
male  and  female  Soldiers,  including  upper  and  lower  bounds  based  upon  the  standard 
error  of  the  estimated  ST  coefficient.  For  a  particular  ST  score,  trace  a  vertical  line  up  to 
the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the  corresponding 
probability  of  passing.  For  example,  an  ST  score  of  100  corresponds  to  a  passing 
probability  of  about  95%  for  a  high  school  educated,  Regular  Army  male  Soldier.  One 
can  see  that  the  probability  of  passing  increases  slowly  for  male  soldiers  as  ST  increases, 
while  it  increases  more  rapidly  for  female  soldiers. 
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Probability  of  pasing 
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Appendix:  Soldier  Characteristics  (25V) 


Gender 

Outcome 

Male 

Female  | 

fail 

3.7% 

12.5% 

pass 

96.3% 

87.5% 

Education  Level  j 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

.0% 

8.5% 

.0% 

pass 

100.0% 

91 .5% 

100.0% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

10.0% 

.0% 

.0% 

pass 

90.0% 

100.0% 

100.0% 
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25C  (31C):  Radio  Operator  -  Maintainer 


The  final  sample  included  1793  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  for  this  MOS  during  a  period  from  2001 
to  2004.  Only  Soldiers  who  graduated  (99.2%)  or  failed  for  academic  reasons  (0.8%) 
were  included  in  the  analysis  sample  of  1793  Soldiers  because  we  assume  that  non¬ 
completion  for  non-academic  reasons  is  not  related  to  cognitive  criteria. 

In  this  sample,  most  were  male  (92%,  8%  female),  most  had  a  high  school 
diploma  but  not  more  education  (82%,  8%  some  college,  10%  GED  or  less),  and  most 
were  from  were  from  the  National  Guard  (57%,  42%  Regular  Army,  1%  Reserve).  The 
governing  AA  composites,  Electrical  Repair  (EL)  and  Surveillance/Communications 
(SC),  for  this  MOS  both  have  cutoff  scores  of  98;  the  sample  mean  is  1 1 1.99  (standard 
deviation  =  9.298)  for  EL  and  1 1.84  (standard  deviation  =  9.081)  for  SC.  EL  and  SC  are 
correlated  at  .864  (p  <  .001).  See  the  Appendix  for  a  description  of  Soldier 
characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 


Table  1.  3 1C:  Results  of  the  forward  stepwise 
binary  logistic  prediction  model _ 


Chi-  Square 

7.773* 

Log  Likelihood 

165.609 

Nagelkerke  R  Square 

.04 

17 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

6.548 

697.563 

3.669 

GUARD 

1.418 

4.128 

5.696* 

EL 

-.076 

.927 

2.052 

SC 

.055 

1.057 

1.026 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 
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outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  binary  logistic  model,  including  the  EL  and  SC  composites, 
and  Army  component  (GUARD  =  National  Guard),  accounts  for  five  percent  of  the 
variation  in  the  dependent  variable.  This  model  is  statistically  significant  (%  =  7.773,  P  = 
.051),  and  has  limited  explanatory  power  (Nagelkerke  R"  =  .047). 

There  are  no  statistically  significant  effects  for  either  the  EL  or  SC  composites, 
likely  due  to  the  small  number  of  failure  cases.18  There  is  a  statistically  significant  effect 
for  GUARD,  with  increased  odds  of  passing  the  course  at  313%. 

Policy  Analysis.  Given  the  model’s  lack  of  explanatory  power,  and 
notwithstanding  the  statistically  significant  effect  for  National  Guard  membership,  this 
model  is  not  appropriate  for  conducting  policy  analysis. 


18  It  is  noteworthy  that  the  coefficient  is  negative  for  EL  and  positive  for  SC.  Given  the  high  correlation 
between  the  two,  such  a  relationship  usually  suggests  collinearity.  The  Appendix  presents  three  additional 
models  to  test  for  this.  When  either  EL  (Table  la)  or  SC  (Table  lb)  is  included  as  the  only  composite,  the 
EL  or  SC  coefficient  is  negative.  Finally,  a  forward  stepwise  method  binary  logistic  regression  model  was 
estimated  (Table  lc).  This  model  excluded  both  EL  and  SC,  leaving  National  Guard  membership  as  the 
sole  predictor  of  training  success  in  MOS  3 1C.  These  results  do  not  lend  support  to  collinearity. 
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Appendix:  Additional  Regression  Models 


Table  lb.  3 1C:  Results  of  the  binary  logistic 
prediction  model  excluding  EL 

Chi-  Square 

5.832 

Log  Likelihood 

167.550 

Nagelkerke  R  Square 

.035 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

5.436 

229.586 

2.775 

GUARD 

1.322 

3.750 

5.046* 

SC 

-.011 

.989 

.137 

*  =  p  <  .05  ***  =  p<.00] 

**  =  p  <  .01 

[ 

Table  lc.  3 1C:  Results  of  the  forward  stepwise 
binary  logistic  prediction  model 

Chi-  Square 

5.693* 

Log  Likelihood 

167.689 

Nagelkerke  R  Square 

.034 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

4.237 

69.182 

194.636*** 

GUARD 

1.302 

3.675 

4.936 

*  =  p  <  .05  ***  =  p  <  .00] 

**  =  p  <  .01 

[ 
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Appendix:  Soldier  Characteristics  (3 1C) 


Gender 

Outcome 

Male 

Female  [ 

fail 

.8% 

pass 

99.2% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

1.1% 

.9% 

.0% 

pass 

98.9% 

99.1% 

100.0% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

1.5% 

.0% 

.4% 

pass 

98.5% 

100.0% 

99.6% 
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25F  (31F):  Network  Switching  Systems  Operator 


The  final  sample  included  3872  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  for  this  MOS  during  a  period  from  2001 
to  2004.  Only  Soldiers  who  graduated  (91.9%)  or  failed  for  academic  reasons  (8.1%) 
were  included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  this  sample,  most  were  male  (88%,  12%  female),  most  had  a  high  school 
diploma  but  not  more  education  (84%,  13%  some  college,  3%  GED  or  less),  and  most 
were  from  were  from  the  Regular  Anny  (84%,  13%  National  Guard,  3%  Reserve).  The 
governing  AA  composites,  Electrical  Repair  (EL)  and  Surveillance/Communications 
(SC),  for  this  MOS  have  cutoff  scores  of  102  and  105  respectively;  the  sample  mean  is 
1 14.87  (standard  deviation  =  8.570)  for  EL  and  1 14.71  (standard  deviation  =  8.069)  for 
SC.  The  EL  and  SC  composites  are  correlated  at  the  .842  level  (p  <  .001).  See  the 
Appendix  for  a  description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 


Table  1.  3  IF:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

64.574*** 

Log  Likelihood 

2109.837 

Nagelkerke  R  Square 

.038 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-3.047 

.048 

9.958** 

COLLEGE 

.345 

1.411 

2.774 

GUARD 

1.038 

2.524 

4  964*  *  * 

GENDER 

.321 

1.378 

2.660 

EL 

.003 

1.003 

.058 

SC 

.044 

1.045 

9  494*** 

*  =  p  <  .05  ***=p<.001 

**  =p  <  .01 
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outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  stepwise  model,  including  the  EL  and  SC  composites, 
education  status  (COLLEGE  =  some  college),  Army  component  (GUARD  =  National 
Guard),  and  gender  (GENDER  =  female),  accounts  for  four  percent  of  the  variation  in  the 
dependent  variable.  This  model  is  statistically  significant  (%2  =  64.574,  P  <  .001),  and  has 
limited  explanatory  power  (Nagelkerke  R2  =  .038). 

There  is  a  statistically  significant  effect  for  the  SC  composite.19  The  model 
suggests  that  having  a  higher  SC  composite  score  increases  the  odds  of  passing.  At  the 
mean  SC  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  5%  in  the 
odds  of  passing  the  course,  and  a  five-point  increase  in  SC  would  increase  the  odds  of 
passing  by  25%.  There  is  also  a  statistically  significant  effect  for  GUARD,  with 
increased  odds  of  passing  the  course  at  182%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

Q(P’x>  /  (1  +  Q(P’x)y 

The  following  analysis  is  focused  upon  male  Soldiers  with  a  high  school  diploma  from 
the  Regular  Army  (the  modal  categories).  Based  on  the  model,  when  the  cutoff  score  is 
at  its  current  level  (EL  =  102,  SC  =  105),  male  Soldiers  with  a  high  school  diploma  and 
average  SC  and  EL  scores  (SC  =  1 15.68,  EL  =  1 15.68)  have  approximately  a  92%  chance 

19  In  the  Appendix  we  also  report  estimated  models  excluding  EL  (Table  la)  and  SC  (Table  lb),  because 
the  aforementioned  correlation  between  the  two  and  the  small  coefficient  for  the  EL  composite  suggested 
collinearity.  While  these  models  did  confirm  collinearity  between  EL  and  SC,  neither  model  substantially 
improved  upon  the  one  including  both  composites.  Thus,  the  full  model  was  selected  with  the 
understanding  that  there  is  underlying  collinearity. 
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of  passing.  Currently,  about  46%  of  Soldiers  are  eligible  for  MOS  3  IF  assignment  at  the 
current  cutoff  (EL  =  102,  SC  =  105).  Lowering  the  cutoffs  by  five  points  (EL  =  97,  SC  = 
100)  would  increase  eligibility  by  15  percentage  points  (to  61%),  and  the  average  male 
Soldier  who  would  qualify  for  training  would  have  essentially  the  same  chance  of  passing 
(91%).  Raising  the  cutoff  scores  (EL  =  107,  SC  =  1 10),  the  average  male  Soldier  would 
have  essentially  the  same  chance  of  passing  (93%)  but  eligibility  would  fall  by  about  14 
percentage  points. 


Table  2.  3  IF:  Probabi 
population)  will  pass  t 

iity  that  a  Regular  Anny  Soldier  (from  the  larger  Army  contract 
le  course  based  on  the  binary  logistic  model 

Cutoff: 

EL  =  92 
SC  =  95 

Cutoff: 

EL  =  97 
SC  =  100 

Cutoff 

EL  =  102 
SC  =  105 

Cutoff 

EL  =  107 
SC  =  110 

Cutoff 

EL  =  112 
SC=  115 

Percent  Eligible 
( Regular  Army) 

73.8% 

60.6% 

45.8% 

32.3% 

20.7% 

Mean:  SC 

110.52 

113.33 

116.18 

119.42 

122.57 

Mean:  EL 

110.18 

112.82 

115.68 

118.93 

112.23 

Passing  Rates: 

High  School  Male 

89.5% 

90.7% 

91.8% 

92.9% 

93.8% 

Figure  1  shows  the  relationship  between  SC  and  the  probability  of  passing  for 
male  Regular  Army  Soldiers  based  upon  Table  lb  (Appendix);  it  includes  upper  and 
lower  bounds  based  upon  the  standard  error  of  the  estimated  SC  coefficient.  For  a 
particular  SC  score,  trace  a  vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to 
the  axis  to  determine  the  corresponding  probability  of  passing.  For  example,  an  SC  score 
of  100  corresponds  to  a  passing  probability  of  about  85%  for  a  high  school  educated, 
Regular  Anny  male  Soldier.  One  can  see  that  Soldiers  with  low  SC  scores  stand  a  fair 
chance  of  passing  the  course.  As  SC  scores  increase,  the  chance  that  a  Soldier  will  pass 
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Probability  of  passing 


the  course  increases.  By  the  time  the  SC  score  reaches  the  current  cutoff  (SC  =  105), 
male  Soldiers  already  have  approximately  an  85%  chance  of  passing  the  course. 


Figure  1:  Predicted  Probability  of  Passing:  Male  Regular  Army 
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Appendix:  Additional  Regression  Models. 


Table  la.  3  IF:  Results  of  the  binary  logistic 
prediction  model  excluding  SC _ 


Chi-  Square 

55.157*** 

Log  Likelihood 

2119.254 

Nagelkerke  R  Square 

.033 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-1.953 

.142 

25.669*** 

COLLEGE 

.357 

1.430 

2.995 

GUARD 

1.021 

2.777 

14.731*** 

GENDER 

.288 

1.334 

2.158 

EL 

.037 

1.038 

25.669*** 

*  =  p  <  .05  ***=p<.001 

**  =p  <  .01 


Table  lb.  3  IF:  Results  of  the  binary  logistic 
prediction  model  excluding  EL 

Chi-  Square 

64.516*** 

Log  Likelihood 

2109.895 

Nagelkerke  R  Square 

.038 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-3.000 

.050 

10.113 

COLLEGE 

.347 

1.415 

2.825 

GUARD 

1.039 

2.828 

15.014*** 

GENDER 

.319 

1.376 

2.640 

SC 

.046 

1.047 

30.929*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 

[ 
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Appendix:  Soldier  Characteristics  (3  IF) 


Gender 

Outcome 

Male 

Female  | 

fail 

8.3% 

6.9% 

pass 

91.7% 

93.1% 

Education  Level  j 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

4.7% 

8.6% 

5.5% 

95.3% 

91 .4% 

94.5% 

Branch  of  Army 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

9.0% 

2.3% 

3.4% 

pass 

91.0% 

97.7% 

96.6% 
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25L  (31L):  Cable  Systems  Installer  -  Maintainer 


The  final  sample  included  2131  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  for  this  MOS  during  a  period  from  2001 
to  2004.  Only  Soldiers  who  graduated  (99.3%)  or  failed  for  academic  reasons  (0.7%) 
were  included  in  the  analysis  sample  of  2131  Soldiers  because  we  assume  that  non¬ 
completion  for  non-academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (76%,  24%  female),  most  had  a  high  school 
diploma  but  not  more  education  (87%,  7%  some  college,  6%  GED  or  less),  and  most 
were  from  were  from  the  Regular  Anny  (56%,  28%  National  Guard,  16%  Reserve).  The 
governing  AA  composites,  Electrical  Repair  (EL)  and  Surveillance/Communications 
(SC),  for  this  MOS  have  cutoff  scores  of  89  and  90;  the  sample  mean  is  100.85  (standard 
deviation  =  10.446)  for  EL  and  100.74  (standard  deviation  =  10.184)  for  SC.  EL  and  SC 
are  correlated  at  .871  (p  <  .001).  See  the  Appendix  for  a  description  of  Soldier 
characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 


Table  1.  31L:  Results  of  the  forward  stepwise 
binary  logistic  prediction  model _ 


Chi-  Square 

2.333 

Log  Likelihood 

176.250 

Nagelkerke  R  Square 

.014 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

4.081 

59.223 

2.029 

GENDER 

-.656 

.519 

1.468 

EL 

-.027 

.973 

.338 

SC 

.038 

1.039 

.565 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 
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Table  1  and  indicate  that  the  binary  logistic  model,  including  the  EL  and  SC  composites, 
and  gender  (GENDER  =  female),  accounts  for  one  percent  of  the  variation  in  the 
dependent  variable.  This  model  is  not  statistically  significant  (y2  =  2.333,  P  =  .506),  and 
has  little  explanatory  power  (Nagelkerke  R"  =  .014). 

There  are  no  statistically  significant  effects  for  the  EL  or  SC  composites.  The 
data  are  insufficient  (too  few  failure  observations)  to  determine  if  any  of  the  variables, 
including  the  composites,  have  a  consistent  effect  upon  passing. 

Policy  Analysis.  Given  the  model’s  lack  of  explanatory  power,  this  model  is  not 
appropriate  for  conducting  policy  analysis. 
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Appendix:  Additional  Regression  Models 


Table  la.  31L:  Results  of  the  binary  logistic 
prediction  model  excluding  SC _ 


Chi-  Square 

6.776* 

Log  Likelihood 

116.606 

Nagelkerke  R  Square 

.04 

1 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

7.563 

1926.569 

5.486* 

GUARD 

1.384 

3.989 

5.470* 

EL 

-.030 

.971 

1.093 

*  =  p  <  .05  ***  =  p<.001 

**  =  p  <  .01 


Table  lb.  31L:  Results  of  the  binary  logistic 
prediction  model  excluding  EL 

Chi-  Square 

5.832 

Log  Likelihood 

167.550 

Nagelkerke  R  Square 

.035 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

5.436 

229.586 

2.775 

GUARD 

1.322 

3.750 

5.046* 

SC 

-.011 

.989 

.137 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 

[ 

Table  lb.  31L:  Results  of  the  forward  stepwise 
binary  logistic  prediction  model 

Chi-  Square 

5.693* 

Log  Likelihood 

167.689 

Nagelkerke  R  Square 

.034 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

4.237 

69.182 

194.636*** 

GUARD 

1.302 

3.675 

4.936 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 

l 
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Appendix:  Soldier  Characteristics  (31L) 


Gender 

Outcome 

Male 

Female  | 

fail 

.6% 

pass 

99.4% 

Education  Level  I 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

.8% 

.8% 

.0% 

pass 

99.2% 

99.2% 

100.0% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

.8% 

.3% 

.7% 

pass 

99.2% 

99.7% 

99.3% 
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25P  (3 IP):  Microwave  Systems  Operator  -  Maintainer 


The  final  sample  included  755  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  for  this  MOS  during  a  period  from  2001 
to  2004.  Only  Soldiers  who  graduated  (93.6%)  or  failed  for  academic  reasons  (6.4%) 
were  included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (92%,  8%  female),  most  had  a  high  school 
diploma  but  not  more  education  (82%,  17%  some  college,  2%  GED  or  less),  and  most 
were  from  were  from  the  Regular  Anny  (91%,  6%  National  Guard,  3%  Reserve).  The 
governing  AA  composites,  Electrical  Repair  (EL),  for  this  MOS  has  a  cutoff  score  of 
107;  the  sample  mean  is  1 16.73  (standard  deviation  =  7.007).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  binary  logistic  model,  including  the  EL  composite, 


Table  1.  3  IP:  Results  of  the  forward  stepwise 
binary  logistic  prediction  model _ 


Chi-  Square 

4.626 

Log  Likelihood 

352.786 

Nagelkerke  R  Square 

.016 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-.236 

.789 

.008 

COLLEGE 

.242 

1.274 

.284 

GENDER 

-.702 

.496 

2.554 

EL 

.026 

1.026 

1.196 

*  =  p  <  .05  ***  =  p  <  .001 

**  =p  <  .01 
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education  status  (COLLEGE  =  some  college),  and  gender  (GENDER  =  female),  accounts 
for  one  percent  of  the  variation  in  the  dependent  variable.  This  model  is  not  statistically 
significant  (x  =  4.626,  P  =  .201),  and  has  little  explanatory  power  (Nagelkerke  R  = 
.016). 

There  are  no  statistically  significant  effects  for  the  EL  composite.  The  data  are 
insufficient  (too  few  failure  observations)  to  determine  if  any  of  the  variables,  including 
EL,  have  a  consistent  effect  upon  passing. 

Policy  Analysis.  Given  the  model’s  lack  of  explanatory  power,  this  model  is  not 
appropriate  for  conducting  policy  analysis. 


129 


Appendix:  Soldier  Characteristics  (3  IP) 


Gender 

Outcome 

Male 

Female  [! 

fail 

5.9% 

12.1% 

pass 

94.1% 

87.9% 

Education  Level  I 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

7.7% 

6.6% 

4.8% 

pass 

92.3% 

93.4% 

95.2% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

6.8% 

.0% 

pass 

93.2% 

100.0% 
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25R  (31R):  Multi-channel  Transmissions  Systems  Operator  -  Maintainer 


The  final  sample  included  5849  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  for  this  MOS  during  a  period  from  2001 
to  2004.  Only  Soldiers  who  graduated  (99.2%)  or  failed  for  academic  reasons  (0.8%) 
were  included  in  the  analysis  sample  of  5849  Soldiers  because  we  assume  that  non¬ 
completion  for  non-academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (86%,  14%  female),  most  had  a  high  school 
diploma  but  not  more  education  (88%,  10%  some  college,  2%  GED  or  less),  and  most 
were  from  were  from  the  Regular  Anny  (84%,  13%  National  Guard,  3%  Reserve).  The 
governing  AA  composites,  Electrical  Repair  (EL)  and  Surveillance/Communications 
(SC),  for  this  MOS  both  have  cutoff  scores  of  98;  the  sample  mean  is  1 10.01  (standard 
deviation  =  8.968)  for  EL  and  109.97  (standard  deviation  =  8.656)  for  SC.  EL  and  SC  are 
correlated  at  .861  (p  <  .001).  See  the  Appendix  for  a  description  of  Soldier 
characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 


Table  1.  31R:  Results  of  the  forward  stepwise 
binary  logistic  prediction  model _ 


Chi-  Square 

8.356 

Log  Likelihood 

519.359 

Nagelkerke  R  Square 

.017 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

1.113 

3.043 

.700 

COLLEGE 

.294 

1.341 

.236 

RESERVE 

-.921 

.398 

2.307 

GENDER 

-.202 

.817 

.264 

EL 

.021 

1.021 

.360 

SC 

.014 

1.014 

.165 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 
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outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  binary  logistic  model,  including  the  EL  and  SC  composites, 
education  level  (COLLEGE  =  some  college)  and  Army  component  (RESERVE  =  Army 
Reserve),  accounts  for  two  percent  of  the  variation  in  the  dependent  variable.  This  model 
is  not  statistically  significant  (y2  =  8.356,  P  =  .138),  and  has  little  explanatory  power 
(Nagelkerke  R2  =  .017). 

There  are  no  statistically  significant  effects  for  either  the  EL  or  the  SC  composite. 
The  data  are  likely  insufficient  (too  few  failure  observations)  to  detennine  if  any  of  the 
variables,  including  the  composites,  have  a  consistent  effect  upon  passing. 

Policy  Analysis.  Given  the  model’s  lack  of  explanatory  power,  this  model  is  not 
appropriate  for  conducting  policy  analysis. 


132 


Appendix:  Soldier  Characteristics  (31R) 


Gender 

Outcome 

Male 

Female  | 

fail 

7.6% 

6.7% 

pass 

92.4% 

93.3% 

Education  Level  j 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

.0% 

8.4% 

5.5% 

pass 

100.0% 

91 .6% 

94.5% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

.0% 

.0% 

100.0% 

100.0% 
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25S  (3 IS):  Satellite  Communications 


The  final  sample  included  1236  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  for  this  MOS  during  a  period  from  2001 
to  2004.  Only  Soldiers  who  graduated  (92.4%)  or  failed  for  academic  reasons  (7.6%) 
were  included  in  the  analysis  sample  of  1236  Soldiers  because  we  assume  that  non¬ 
completion  for  non-academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (96%,  4%  female),  most  had  a  high  school 
diploma  but  not  more  education  (76%,  24%  some  college,  1%  GED  or  less),  and  most 
were  from  were  from  the  Regular  Anny  (99%,  <  1%  National  Guard,  <  1%  Reserve). 

The  governing  AA  composite,  Electrical  Repair  (EL),  for  this  MOS  has  a  cutoff  scores  of 
1 16;  the  sample  mean  is  123.85  (standard  deviation  =  5.606).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  binary  logistic  model,  including  the  EL  the  composite, 


Table  1.  3  IS:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

5.277 

Log  Likelihood 

659.738 

Nagelkerke  R  Square 

.010 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-1.383 

.251 

.349 

COLLEGE 

.394 

1.483 

1.910 

GENDER 

.176 

1.192 

.082 

EL 

.031 

1.031 

2.604 

*  =  p  <  .05  ***=p<.001 

**  =p  <  .01 
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education  level  (COLLEGE  =  some  college),  and  gender  (GENDER  =  female),  accounts 
for  one  percent  of  the  variation  in  the  dependent  variable.  This  model  is  not  statistically 
significant  (x  =  5.277,  P  =  .153),  and  has  little  explanatory  power  (Nagelkerke  R  = 
.010). 

There  are  no  statistically  significant  effects  for  the  EL  composite.  Given  the 
relatively  high  cutoff  score  level  on  the  governing  composite,  we  must  presume  there  are 
other  factors  not  captured  in  the  model  that  would  differentiate  between  passing  and 
failure  cases. 

Policy  Analysis.  Given  the  model’s  lack  of  explanatory  power,  this  model  is  not 
appropriate  for  conducting  policy  analysis. 
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Appendix:  Soldier  Characteristics  (3 IS) 


Gender 

Male 

Female 

Column  N  % 

Column  N  % 

s1_pf1  fail 

7.6% 

6.7% 

pass 

92.4% 

93.3% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

Column  N  % 

Column  N  % 

Column  N  % 

s1_pf1 

fail 

.0% 

8.4% 

5.5% 

pass 

100.0% 

91 .6% 

94.5% 

Component 

Army 

National 

Regular  Army 

Reserve 

Guard 

Column  N  % 

Column  N  % 

Column  N  % 

s1_pf1  fail 

.0% 

.0% 

pass 

100.0% 

100.0% 

136 


25U  (31U):  Signal  Support  Systems  Specialist 


The  final  sample  included  5568  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  for  this  MOS  during  a  period  from  2001 
to  2004.  Only  Soldiers  who  graduated  (96.9%)  or  failed  for  academic  reasons  (3.1%) 
were  included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  this  sample,  most  were  male  (83%,  17%  female),  most  had  a  high  school 
diploma  but  not  more  education  (87%,  8%  some  college,  6%  GED  or  less),  and  most 
were  from  were  from  the  Regular  Anny  (64%,  27%  National  Guard,  9%  Reserve).  The 
governing  AA  composites,  Electrical  Repair  (EL)  and  Surveillance/Communications 
(SC),  for  this  MOS  both  have  cutoff  scores  of  93;  the  sample  mean  is  106.05  (standard 
deviation  =  10.417)  for  EL  and  105.61  (standard  deviation  =  10.041)  for  SC.  EL  and  SC 
are  correlated  at  0.872  (p  <  .001).  See  the  Appendix  for  a  description  of  Soldier 
characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 


Table  1.  31U:  Results  of  the  forward  stepwise 
binary  logistic  prediction  model  excluding  SC 


Chi-  Square 

39797*** 

Log  Likelihood 

1488.189 

Nagelkerke  R  Square 

.030 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-.397 

.673 

.312 

RESERVE 

.229 

1.257 

.595 

GUARD 

.621 

1.860 

8.509** 

GENDER 

-.055 

.946 

.082 

EL 

.036 

1.036 

27.171*** 

*  =  p  <  .05  ***  =  p<.001 

**  =  p  <  .01 
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outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  stepwise  model,  including  the  EL  composite.  Army 
component  (GUARD  =  National  Guard;  RESERVE  =  Army  Reserve),  and  gender 
(GENDER  =  female),  accounts  for  three  percent  of  the  variation  in  the  dependent 
variable.  This  model  is  statistically  significant  (%2  =  39.797,  P  <  .001),  and  has  limited 
explanatory  power  (Nagelkerke  R2  =  .030). 

There  is  a  statistically  significant  effect  for  the  EL  composite.20  The  model 
suggests  that  having  a  higher  EL  composite  score  increases  the  odds  of  passing.  At  the 
mean  EL  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  4%  in  the 
odds  of  passing  the  course,  and  a  five-point  increase  in  EL  would  increase  the  odds  of 
passing  by  20%.  There  is  also  a  statistically  significant  effect  for  GUARD,  with 
increased  odds  of  passing  the  course  at  86%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

q(P’x)  /  (1  +  e(p’x}). 

The  following  analysis  is  focused  upon  male  Soldiers  with  a  high  school  diploma  from 
the  Regular  Army  (the  modal  categories).  Based  on  the  model,  when  the  cutoff  score  is 
at  its  current  level  (EL  =  93,  SC  =  93),  male  Soldiers  with  a  high  school  diploma  and  an 

20  initially  we  estimated  a  model  including  EL,  SC,  GUARD,  RESERVE,  and  GENDER.  This  is  reported 
in  the  Appendix  (Table  la).  In  that  model  neither  of  the  coefficients  for  EL  or  SC  was  significant.  Because 
the  SC  coefficient  was  close  to  being  significant,  it  was  theorized  that  the  lack  of  significance  was  due  to 
collinearity.  The  stepwise  model  retained  EL  while  excluding  SC,  and  thus  confirmed  this  presumption.  As 
further  evidence,  we  found  a  statistically  significant  effect  for  the  SC  composite  when  inserted  in  place  of 
the  EL  composite;  see  the  Appendix  (Table  lb). 
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average  EL  score  (EL  =  1 10.00)  have  approximately  a  97%  chance  of  passing.  Currently, 
about  76%  of  Soldiers  are  eligible  for  MOS  3 1U  assignment  at  the  current  cutoff  (EL  = 
93,  SC  =  93).  Lowering  the  cutoffs  by  five  points  (EL  =  88,  SC  =  88)  would  increase 
eligibility  by  12  percentage  points  (to  88%),  and  the  average  male  Soldier  who  would 
qualify  for  training  would  have  essentially  the  same  chance  of  passing  (97%).  Raising 
the  cutoff  scores  (EL  =  98,  SC  =  98),  the  average  male  Soldier  would  have  essentially  the 


Table  2.  31U:  Probata 
population)  will  pass  t 

lity  that  a  Regular  Anny  Soldier  (from  the  larger  Army  contract 
le  course  based  on  the  binary  logistic  model 

Cutoff: 

EL  =  83 
SC  =  83 

Cutoff: 

EL  =  88 
SC  =  88 

Cutoff 

EL  =  93 
SC  =  93 

Cutoff 

EL  =  98 
SC  =  98 

Cutoff 

EL  =  103 
SC  =  103 

Percent  Eligible 
(Regular  Army) 

96.1% 

88.2% 

75.9% 

62.5% 

47.7%o 

Mean 

106.23 

107.73 

110.00 

112.56 

115.55 

Passing  Rates: 

High  School  Male 

96.9% 

97.1% 

97.3% 

97.5% 

97.8% 

same  chance  of  passing  (98%)  but  eligibility  would  fall  by  about  13  percentage  points. 

Ligure  1  shows  the  relationship  between  EL  and  the  probability  of  passing  for 
male  Regular  Army  Soldiers  by  education,  including  upper  and  lower  bounds  based  upon 
the  standard  error  of  the  estimated  EL  coefficient.  Lor  a  particular  EL  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
corresponding  probability  of  passing.  Lor  example,  an  EL  score  of  100  corresponds  to  a 
passing  probability  of  just  above  95%  for  a  high  school  educated,  Regular  Army  male 
Soldier.  One  can  see  that  Soldiers  with  a  low  EL  score  still  stand  a  good  chance  of 
passing  the  course.  As  EL  scores  increase,  the  chance  that  a  Soldier  will  pass  the  course 
increases  somewhat.  However,  by  the  time  EL  scores  reach  the  current  cutoff  (EL  =  93) 
male  Soldiers  already  have  approximately  a  95%  chance  of  passing  the  course. 
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Probability  of  passing 


Figure  1:  Predicted  Probability  of  Passing:  Male  Regular  Army 


Appendix:  Additional  Regression  Models 


Table  la.  31U:  Results  of  the  binary  logistic 
prediction  model  including  EL  and  SC 


Chi-  Square 

40.059** 

Log  Likelihood 

1487.927 

Nagelkerke  R  Square 

.030 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-.487 

.614 

.445 

GUARD 

.621 

1.860 

8.494** 

RESERVE 

.224 

1.251 

.568 

GENDER 

-.052 

.949 

.072 

EL 

.028 

1.008 

.271 

SC 

.028 

1.029 

3.379 

*  =  p  <  .05  ***  =  p  <  .001 

**  =p  <  .01 


Table  lb.  31U:  Results  of  the  binary  logistic 
prediction  model  excluding  EL 

Chi-  Square 

36.287*** 

Log  Likelihood 

1491.699 

Nagelkerke  R  Square 

.027 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-.124 

.883 

.032 

GUARD 

.624 

1.866 

8.638** 

RESERVE 

.227 

1.255 

.586 

GENDER 

-.073 

.929 

.145 

SC 

.033 

1.034 

24.569*** 

*  =  p  <  .05  ***  =  p  <  .00 1 

**  =  p  <  .01 

[ 
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Appendix:  Soldiers  Characteristics  (31U) 


Gender  1 

Outcome 

Male 

Female  [ 

fail 

2.9% 

3.8% 

pass 

97.1% 

96.2% 

Education  Status  1 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

1.9% 

.9% 

pass 

98.1% 

99.1% 

Component  j 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

3.7% 

2.5% 

pass 

96.3% 

97.5% 
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27D  (71D):  Legal  Specialist 


The  final  sample  included  423  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  for  this  MOS  during  a  period  from  2001 
to  2004.  Only  Soldiers  who  graduated  (88.9%)  or  failed  for  academic  reasons  (11.1%) 
were  included  in  the  analysis  sample  of  423  Soldiers  because  we  assume  that  non¬ 
completion  for  non-academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (55%,  45%  female),  most  had  a  high  school 
diploma  but  not  more  education  (78%,  21%  some  college,  1%  GED  or  less),  and  most 
were  from  were  from  the  Regular  Anny  (63%,  21%  National  Guard,  16%  Reserve).  The 
governing  AA  composites,  Clerical  (CL),  for  this  MOS  has  cutoff  score  of  1 10;  the 
sample  mean  is  1 16.91  (standard  deviation  =  7.242).  See  the  Appendix  for  a  description 
of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  model,  including  the  CL  composite,  education  status 
(COLLEGE  =  some  college  education),  Army  component  (GUARD  =  National  Guard, 
RESERVE  =  Army  Reserve),  and  GENDER  (GENDER  =  Female)  accounts  for  about 
16%  of  the  variation  in  the  dependent  variable.  This  model  is  statistically  significant  (y2  = 
35.844,  P  <  .001),  and  has  moderate  explanatory  power  (Nagelkerke  R"  =  .162). 
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Table  1.  7  ID:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

35.844*** 

Log  Likelihood 

259.268 

Nagelkerke  R  Square 

.162 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-15.010 

.000 

15.790*** 

COLLEGE 

1.124 

3.368 

4.872* 

GUARD 

1.426 

4.163 

7.892** 

RESERVE 

.558 

1.747 

1.358 

GENDER 

.676 

1.967 

4.022* 

CL 

.142 

1.152 

18  777*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =p  <  .01 


There  is  a  statistically  significant  effect  for  the  CL  composite.  The  model 
suggests  that  having  a  higher  CL  composite  score  increases  the  odds  of  passing.  At  the 
mean  CL  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  15%  in 
the  odds  of  passing  the  course,  and  a  five-point  increase  in  CL  would  increase  the  odds  of 
passing  by  103%.  There  are  also  statistically  significant  effects  for  COLLEGE,  GUARD, 
and  GENDER,  with  increased  odds  of  passing  the  course  at  237%,  316%,  and  97%  (for 
females),  respectively. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

q(P'x)  /  (1  +  q(P’x>). 

The  following  analysis  is  focused  upon  male  and  female  Soldiers  with  a  high  school 
diploma  or  some  college  education  from  the  Regular  Army.  Based  on  the  model,  when 
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the  cutoff  score  is  at  its  current  level  (CL  =  110),  male  (female)  Soldiers  with  a  high 
school  diploma  and  an  average  CL  score  (CL  =  1 18.71)  have  approximately  an  86% 
(93%)  chance  of  passing.  Currently,  about  36%  of  Soldiers  are  eligible  for  MOS  7  ID 
assignment  at  the  current  cutoff.  Lowering  the  cutoff  by  five  points  (CL  =  105)  would 


Table  2.  7 ID:  Probability  that  a  Regular  Army  Soldier  (from  the  larger  successful 
applicant  population)  will  pass  the  course  based  on  the  binary  logistic  model _ 


Cutoff: 

Cutoff: 

Cutoff 

Cutoff 

Cutoff 

CL  =  100 

CL  =  105 

CL  =  110 

CL  =  115 

CL  =  120 

Percent  Eligible 
(Regular  Army) 

67.6% 

51.2% 

35.7% 

23.2%> 

13.4% 

Mean 

112.63 

115.40 

118.71 

122.06 

125.38 

Passing  Rates: 

High  School  Male 

72.8% 

79.8% 

86.4% 

91.1% 

94.2% 

High  School  Female 

84.0% 

88.6% 

92.6% 

95.3% 

97.0% 

College  Male 

90.0% 

93.0% 

95.5% 

97.2% 

98.2% 

College  Female 

94.7% 

96.3% 

97.7% 

98.5% 

99.1% 

increase  eligibility  by  15  percentage  points  (to  51%)  and  the  average  male  (female) 
Soldier  who  would  qualify  for  training  would  have  a  lower  chance  of  passing  (80%) 
(89%).  Raising  the  CL  cutoff  score  by  five  points  (CL  =110)  leads  to  an  increase  in 
passing  rates  for  the  average  high  school  male  (female)  Soldier  (91%)  (95%),  but  few 
Soldiers  would  be  eligible  (23%). 

Figure  1  shows  the  relationship  between  CL  and  the  probability  of  passing  for 
high  school  educated  Regular  Army  Soldiers  by  gender.  For  a  particular  CL  score,  trace 
a  vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  detennine  the 
corresponding  probability  of  passing.  For  example,  a  CL  score  of  100  corresponds  to  a 
passing  probability  of  about  30%  for  a  high  school  educated,  Regular  Army  male  Soldier. 
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Chance  of  Passing 


Figure  1:  Predicted  Probability  of  Passing:  High  School  Soldiers 


^^^High  School  Male  High  School  Female 


Figure  2  shows  the  relationship  between  CL  and  the  probability  of  passing  for 
male  Soldiers  by  education  level.  The  graph  illustrates  that  college  educated  Soldiers 
have  a  noticeably  better  chance  of  passing  this  course. 
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Figure  2:  Predicted  Probability  of  Passing:  Male  Soldiers 


"  College  Male  ^^^High  School  Male 


Figure  3  shows  the  relationship  between  CL  and  the  probability  of  passing  for 
Soldiers  by  education  and  gender.  The  graph  illustrates  that  women  with  some  college 
education  have  the  best  chance  at  passing  the  course,  while  high  school  educated  males, 
the  modal  students,  have  the  lowest  chance  of  passing.  Figure  3  also  illustrates  the 
magnitude  of  the  effect  brought  about  by  gender  and  education.  At  the  current  cutoff 
score  (CL  =110)  college  educated  female  Soldiers  have  about  a  25  percentage  point 
better  chance  of  passing  the  course  compared  to  high  school  educated  males. 
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Chance  of  Passing 


Figure  3:  Predicted  Probability  of  Passing:  Education  Level  by  Sex 


^^^College  Male  College  Female  High  School  Male  "  ■  High  School  Female 
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Appendix:  Soldier  Characteristics  (7 ID) 


Gender 

Outcome 

Male 

Female 

fail 

12.0% 

10.0% 

pass 

88.0% 

90.0% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

25.0% 

12.8% 

4.4% 

pass 

75.0% 

87.2% 

95.6% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

13.1% 

8.8% 

6.8% 

pass 

86.9% 

91.2% 

93.2% 
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7  ID  Risk  Analysis  of  Policy  Alternatives 


In  this  section  we  extend  the  policy  analysis  with  a  risk  analysis  of  the  policy 
alternatives  considered  using  simulation  methods.  We  employ  the  “@RISK”  software 
package  which  is  designed  as  an  add-in  for  Microsoft  Excel.  There  are  four  steps  in  the 
risk  analysis. 

The  first  step  is  developing  a  model.  A  regression  model  for  predicting  the 
passing  rate  has  been  specified  and  estimated,  and  the  results  stored  in  Excel  worksheet 
format.  See  Table  1. 

The  second  step  is  identifying  the  uncertainty  inherent  in  both  the  input  variables 
(composite  scores,  Soldier  characteristics)  and  the  corresponding  estimated  parameters 
and  specifying  their  possible  values  with  probability  distributions. 

To  approximate  the  governing  composite  distribution,  we  use  the  larger  Army 
contract  population  (EAF  file:  those  who  contracted  from  Jan  1992  to  Aug  2003).  This 
distribution  can  be  assumed  to  follow  a  normal  distribution.  The  mean  and  standard 
deviation  estimated  from  the  contract  population  and  used  as  input  to  the  simulations  are 
shown  in  Table  R-l  (before  truncation). 


Table  R-l.  Input  infonnation  for  the  simulations 

Mean 

Std  error  / 
deviation 

Constant 

-15.010 

3.778 

CL  var 

107.16  (before  truncation) 

11.355 

CL  coeff 

.142 

.033 

COLLEGE  var 

21.3% 

COLLEGE  coeff 

1.214 

.550 

GUARD  var 

20.8% 

GUARD  coeff 

1.426 

.508 

RESERVE  var 

16.1% 

RESERVE  coeff 

.558 

.479 

GENDER  var 

55.1% 

GENDER  coeff 

.676 

.337 

To  approximate  the  distributions  for  the  demographic  variables  we  rely  on  the 
training  data  sample  used  to  estimate  the  regression  model.  These  distributions  can  be 


21  Software  is  available  from  Palisade  Corporation,  Newfield,  NY. 
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assumed  to  follow  binomial  distributions  with  means  estimated  from  the  training  data 
sample  as  shown  in  Table  R-l. 


To  approximate  the  distributions  for  the  estimated  parameters  in  the  logit  model, 
we  note  that,  under  very  general  conditions,  the  estimates  will  be  asymptotically  normally 
distributed.22  The  estimated  coefficients  (means  and  standard  errors)  are  shown  in  Table 


R-l. 


The  third  step  is  analyzing  the  model  with  simulation  to  determine  the  range  and 
probabilities  of  all  possible  training  outcome  results.  The  results  reported  represent  the 
simulation  of  the  regression  prediction  equation  for  1000  iterations  using  a  fixed  seed. 

The  comparisons  are  between  the  baseline  case  (CL  =110)  and  the  policy  cases  (CL  = 

105  and  CL  =  115).  These  cases  are  delineated  by  the  cutoff  score  level  which  serves  as 
the  lower  truncation  point  in  the  governing  composite  input  distribution.  The  simulation 
results  in  Table  R-2  describe  the  effect  on  the  mean  probability  of  passing  as  the  CL 
cutoff  score  level  is  varied,  and  represent  an  improvement  of  the  policy  analysis 
presented  in  Table  2  because  they  reflect  the  simultaneous  effects  of  all  known  sources  of 
uncertainty. 


Table  R-2.  7 ID  simulation  resu' 

Its 

CL  =  105 

CL  =  110 

CL  =  115 

Mean  governing 
composite 

114.88 

118.09 

121.72 

Mean  probability 
of  passing 

65.3% 

68.5% 

70.5% 

Std  deviation 

39.9% 

39.5% 

38.9% 

The  frequency  distributions  of  simulated  effects  are  shown  below  in  the  histogram 
graphs.  The  histogram  portrays  the  full  range  of  outcomes  that  average  to  the  mean 
effect  as  reported.  Looking  at  the  histogram  for  the  baseline  case  (CL  =  110),  probability 
of  passing  is  plotted  on  the  horizontal  axis  and  the  height  of  each  bar  indicates  the 
relative  number  of  Soldiers  at  that  passing  probability.  The  information  in  the  histograms 
is  effectively  summarized  in  Table  R-3.  In  that  table  we  see  the  percentage  of  Soldiers 
who  meet  or  exceed  a  given  chance  of  passing.  For  example,  looking  at  the  baseline  case 
(CL  =  110),  about  64%  of  Soldiers  have  a  70%  or  greater  chance  of  passing. 


22  The  intuition  is  that  logit  estimates  are  derived  from  maximum  likelihood  (ML)  estimation,  and  under 
fairly  general  conditions  ML  estimates  are  consistent  and  asymptotically  normally  distributed. 

23  Latin  Hypercube  (LH)  sampling  is  utilized  in  drawing  samples  for  the  input  distributions.  It  is  designed 
to  accurately  recreate  the  input  distribution  through  sampling  in  fewer  iterations  when  compared  with  the 
Monte  Carlo  method.  The  key  to  LH  sampling  is  stratification  of  the  input  probability  distributions. 
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Table  R-3. 

Percent  of  simulated  Soldiers  that  meet  or  exceed 
given  chance  of  passing 


Chance  of 
passing 

CL  =  105 

CL  =  110 

CL  =  115 

95.0% 

43.6% 

49.3% 

52.2% 

90.0% 

49.8% 

54.6% 

56.3% 

85.0% 

52.9% 

57.4% 

60.7% 

80.0% 

55.1% 

59.9% 

63.0% 

75.0% 

57.7% 

62.1% 

65.1% 

70.0% 

59.4% 

63.9% 

66.4% 

65.0% 

61.4% 

65.6% 

67.5% 

60.0% 

63.2% 

67.1% 

69.1% 

55.0% 

64.9% 

68.5% 

70.5% 

50.0% 

66.7% 

69.5% 

72.2% 

We  find  that  the  mean  probability  of  passing  in  the  simulation  is  69%  (Table  R-2, 
baseline).  Looking  at  the  histograms,  we  see  a  large  portion  of  Soldiers  in  the  highest 
range  of  passing  scores;  however,  it  is  also  evident  that  a  not  insubstantial  group  of 
Soldiers  has  a  very  low  chance  of  passing.  Sensitivity  analysis  indicates  that  we  can 
attribute  this  pattern  to  the  relatively  large  standard  errors  on  the  constant  and  governing 
composite  terms  (specially  the  former)  that  come  into  play  in  the  simulation.  The 
relatively  large  standard  errors  introduce  variability  into  the  passing  probabilities-  .  We 
note  that  the  mean  simulated  passing  probability  turns  out  to  be  substantially  lower  (15 
percentage  points)  than  the  static  prediction  at  the  mean  CL  score  (Table  2).  In  the 
simulated  predictions  we  are  calculating  an  average  over  all  Soldiers  and  taking  into 
account  all  sources  of  uncertainty,  whereas  in  the  static  prediction  we  are  calculating  the 
probability  of  passing  only  at  the  mean  CL  score,  ignoring  the  distribution  of  Soldier 
scores  etc.  as  well  as  uncertainty.  While  recognizing  this  variability  is  important,  we  also 
note  that  the  policy  case  impacts  relative  to  the  baseline  are  the  same  in  both  simulation 
and  static  prediction  modes. 

The  fourth  step  is  making  a  decision  based  on  the  results  provided  and  the 
preferences  of  the  school  proponent.  This  involves  the  evaluation  by  the  school 
proponent  of  the  tradeoff  between  increased  eligibility  and  reduced  passing  probabilities 
(and  vice-versa). 


24  The  large  standard  error  for  the  constant  allows  scores  to  be  more  spread  out  and  allows  for  large 
groupings  at  the  highest  and  lowest  values.  And  since  the  probabilities  of  passing  are  calculated  from  the 
ratio:  e^' x>  /  (1  +  XJ),  as  the  scores  calculated  from  the  equation  become  extreme,  the  ratios  are  more 
likely  to  approach  zero  and  one. 
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Distribution  for  PASS  EQUATION  110 


0  0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9  1 


Distribution  for  PASS  EQUATION  115 


X  <=0  X  <=1 
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Distribution  for  PASS  EQUATION  105 


X  <=0  X  <-' 
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33 W:  Military  Intelligence  Systems  Operator/Integrator 


The  final  sample  included  572  soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  soldiers  who  graduated  (76.2%)  or  failed  for  academic  reasons  (23.8%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (92%,  8%  female),  most  had  a  high  school 
diploma  but  not  more  education  (85%,  14%  some  college,  <  1%  GED  or  less),  and  most 
were  Regular  Army  (97%,  1%  Anny  Reserve,  2%  National  Guard).  The  governing  AA 
composite,  Skilled  Technical  (ST),  for  this  MOS  has  a  cutoff  score  of  1 13;  the  sample 
mean  is  121.84  (standard  deviation  =  6.262).  See  the  Appendix  for  a  description  of 
Soldier  characteristics  for  this  MOS  training  sample. 


Table  1.  33W:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

60.836*** 

Log  Likelihood 

566.632 

Nagelkerke  R  Square 

.151 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-13.436 

.000 

31.675*** 

COLLEGE 

1.295 

3.653 

8.434** 

GENDER 

.156 

1.168 

.183 

ST 

.120 

1.127 

36.284*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 


A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  model,  including  the  ST  composite,  education  status 
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(COLLEGE  =  some  college),  and  gender  (GENDER  =  female),  accounts  for  about  15% 
of  the  variation  in  the  dependent  variable.  This  model  is  statistically  significant  (%2  = 
60.836,  P  <  .001),  with  moderate  explanatory  power  (Nagelkerke  R  =  .151). 

There  are  statistically  significant  effects  for  the  ST  composite,  having  some 
college  education,  and  National  Guard  membership.  The  model  suggests  that  having  a 
higher  ST  composite  score  increases  the  odds  of  passing.  At  the  mean  ST  score,  an 
increase  of  one  point  is  associated  with  an  increase  of  about  13%  in  the  odds  of  passing 
the  course,  and  a  five-point  increase  in  ST  would  increase  the  odds  of  passing  by  82%. 
There  is  a  noteworthy  effect  for  COLLEGE,  where  the  odds  of  a  Soldier  with  some 
college  education  passing  this  training  exceed  that  of  the  average  male  Soldier  by  over 
265%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  fonnula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

Q(P’x>  /  (1  +  Q(P’x>y 

Because  GENDER  was  not  statistically  significant,  this  analysis  is  confined  to  the  modal 
demographics  factored  by  COLLEGE — high  school  educated  male  Soldiers  from  the 
Regular  Army  (though  results  would  be  similar  for  any  demographic  combination). 
Based  on  the  model,  when  the  cutoff  score  is  at  its  current  level  (ST  =  113),  soldiers  with 
a  high  school  diploma  and  an  average  ST  score  (ST  =  120.51)  have  approximately  a  74% 
chance  of  passing,  while  the  college  educated  counterparts  have  approximately  a  92% 
chance  of  passing.  Currently,  about  only  32%  of  soldiers  are  eligible  for  MOS  33W 
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assignment  at  the  current  cutoff  (ST  =  1 13).  Lowering  the  cutoff  by  five  points  (ST  = 
108)  would  increase  eligibility  by  15  percentage  points,  while  the  average  Soldier  who 
would  qualify  for  training  would  have  a  lower  chance  of  passing  (high  school  =  65%, 
college  =  88%).  Raising  the  cutoff  score  by  five  points  (ST  =  118),  the  average  Soldier 
who  would  still  qualify  for  the  MOS  would  have  a  higher  chance  of  passing  (high  school 
=  81%,  college  =  95%),  but  considerably  fewer  Soldiers  would  be  eligible  (20%). 


Table  2.  33W:  Probability  that  a  Soldier  (from  the  larger  Army  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model _ _ _ 


Cutoff  = 

Cutoff = 

Cutoff = 

Cutoff = 

Cutoff = 

103 

108 

113 

118 

123 

Percent  Eligible 
( Regular  Army) 

61.9% 

46.5%> 

31.9% 

19.8% 

10.7% 

Mean 

114.16 

117.18 

120.51 

123.94 

127.39 

Passing  rates: 

High  school  male 

55.6% 

65.2% 

73.6% 

80.8% 

86.4% 

College  male 

84.2% 

88.5% 

92.0% 

94.5% 

96.3% 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
Regular  Army  Soldiers  with  high  school  and  (some)  college  education,  including  upper 
and  lower  bounds  based  upon  the  standard  error  of  the  estimated  ST  coefficient.  For  a 
particular  ST  score,  trace  a  vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to 
the  axis  to  determine  the  corresponding  probability  of  passing.  For  example,  a  ST  value 
of  100  corresponds  to  a  passing  probability  of  about  20%  for  a  high  school  educated  male 
Soldier.  One  can  see  that  the  probability  of  passing  increases  steeply  over  the  full  range 
of  the  data. 
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Figure  1:  Predicted  Probalility  of  Passing:  Male  Soldiers 
With  a  High  School  Diploma  or  Some  College  Education 
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ST  Composite  Scores 


Appendix:  Soldier  Characteristics  (33W) 


Gender 

Outcome 

Male 

Female 

fail 

23.5% 

27.1% 

pass 

76.5% 

72.9% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

.0% 

26.6% 

7.3% 

pass 

100.0% 

73.4% 

92.7% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

24.0% 

16.7% 

16.7% 

pass 

76.0% 

83.3% 

83.3% 
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33W  Risk  Analysis  of  Policy  Alternatives 


In  this  section  we  extend  the  policy  analysis  with  a  risk  analysis  of  the  policy 
alternatives  considered  using  simulation  methods.  We  employ  the  “@RISK”  software 
package  which  is  designed  as  an  add-in  for  Microsoft  Excel.  "  There  are  four  steps  in  the 
risk  analysis. 

The  first  step  is  developing  a  model.  A  regression  model  for  predicting  the 
passing  rate  has  been  specified  and  estimated,  and  the  results  stored  in  Excel  worksheet 
format.  See  Table  1. 

The  second  step  is  identifying  the  uncertainty  inherent  in  both  the  input  variables 
(composite  scores,  Soldier  characteristics)  and  the  corresponding  estimated  parameters 
and  specifying  their  possible  values  with  probability  distributions. 

To  approximate  the  governing  composite  distribution,  we  use  the  larger  Army 
contract  population  (EAF  file:  those  who  contracted  from  Jan  1992  to  Aug  2003).  This 
distribution  can  be  assumed  to  follow  a  normal  distribution.  The  mean  and  standard 
deviation  estimated  from  the  contract  population  and  used  as  input  to  the  simulations  are 
shown  in  Table  R-l  (before  truncation). 


Table  R-l.  Input  infonnation  for  the  simulations 

Mean 

Std  error  / 
deviation 

Constant 

-13.435 

2.387 

ST  var 

107.02  (before  truncation) 

12.157 

ST  coeff 

.120 

.020 

GENDER  var 

8.4%(female) 

GENDER  coeff 

.156 

.363 

COLLEGE  var 

14.3% 

COLEGE  coeff 

1.295 

.446 

_ 

To  approximate  the  distributions  for  the  demographic  variables  (GENDER,  and 
COLLEGE)  we  rely  on  the  training  data  sample  used  to  estimate  the  regression  model. 
These  distributions  can  be  assumed  to  follow  binomial  distributions  with  means 
estimated  from  the  training  data  sample  as  shown  in  Table  R-l. 

To  approximate  the  distributions  for  the  estimated  parameters  in  the  logit  model, 
we  note  that,  under  very  general  conditions,  the  estimates  will  be  asymptotically  normally 


25  Software  is  available  from  Palisade  Corporation,  Newfield,  NY. 
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26 

distributed."  The  estimated  coefficients  (means  and  standard  errors)  are  shown  in  Table 
R-l. 


The  third  step  is  analyzing  the  model  with  simulation  to  determine  the  range  and 
probabilities  of  all  possible  training  outcome  results.  The  results  reported  represent  the 
simulation  of  the  regression  prediction  equation  for  1000  iterations  using  a  fixed  seed. 

The  comparisons  are  between  the  baseline  case  (ST  =113)  and  the  policy  cases  (ST  = 

108  and  ST  =  118).  These  cases  are  delineated  by  the  cutoff  score  level  which  serves  as 
the  lower  truncation  point  in  the  governing  composite  input  distribution.  The  simulation 
results  in  Table  R-2  describe  the  effect  on  the  mean  probability  of  passing  as  the  ST 
cutoff  score  level  is  varied,  and  represent  an  improvement  of  the  policy  analysis 
presented  in  Table  2  because  they  reflect  the  simultaneous  effects  of  all  known  sources  of 
uncertainty. 


Table  R-2.  33W  simulation  results 


09 

H 

II 

o 

CO 

09 

H 

II 

ST=  118 

Mean  governing 
composite 

117.34 

120.81 

124.61 

Mean  probability 
of  passing 

58.0% 

61.9% 

66.1% 

Std  deviation 

37.6% 

37.2% 

36.2% 

The  frequency  distributions  of  simulated  effects  are  shown  below  in  the  histogram 
tables.  The  histogram  portrays  the  full  range  of  outcomes  that  average  to  the  mean  effect 
as  reported.  Looking  at  the  histogram  for  the  baseline  case  (ST  =  113),  probability  of 
passing  is  plotted  on  the  horizontal  axis  and  the  height  of  each  bar  indicates  the  relative 
number  of  Soldiers  at  that  passing  probability.  The  information  in  the  histograms  is 
effectively  summarized  in  Table  R-3.  In  that  table  we  see  the  percentage  of  Soldiers  that 
meet  or  exceed  a  given  chance  of  passing.  For  example,  looking  at  the  baseline  case  (ST 
=  95),  about  53%  of  Soldiers  have  a  70%  or  greater  chance  of  passing. 

We  find  that  the  mean  probability  of  passing  in  the  simulation  is  62%  (Table  R-2, 
baseline).  Looking  at  the  histograms,  we  see  a  large  portion  of  Soldiers  in  the  highest 
range  of  passing  scores;  however,  it  is  also  evident  that  a  not  insubstantial  group  of 
Soldiers  has  a  very  low  chance  of  passing.  Sensitivity  analysis  indicates  that  we  can 
attribute  this  pattern  to  the  relatively  large  standard  errors  on  the  constant  term  and 


2,1  The  intuition  is  that  logit  estimates  are  derived  from  maximum  likelihood  (ML)  estimation,  and  under 
fairly  general  conditions  ML  estimates  are  consistent  and  asymptotically  normally  distributed. 

27  Latin  Hypercube  (LH)  sampling  is  utilized  in  drawing  samples  for  the  input  distributions.  It  is  designed 
to  accurately  recreate  the  input  distribution  through  sampling  in  fewer  iterations  when  compared  with  the 
Monte  Carlo  method.  The  key  to  LH  sampling  is  stratification  of  the  input  probability  distributions. 
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governing  composite  (to  a  lesser  extent)  that  come  into  play  in  the  simulation.  The 
relatively  large  standard  errors  introduce  variability  into  the  passing  probabilities28.  We 
note  that  the  mean  simulated  passing  probability  turns  out  to  be  about  twelve  percentage 
points  lower  than  the  static  prediction  at  the  mean  ST  score  (Table  2).  In  the  simulated 
predictions  we  are  calculating  an  average  over  all  Soldiers  and  taking  into  account  all 
sources  of  uncertainty,  whereas  in  the  static  prediction  we  are  calculating  the  probability 
of  passing  only  at  the  mean  ST  score,  ignoring  the  distribution  of  Soldier  scores  etc.  as 
well  as  uncertainty.  While  recognizing  this  variability  is  important,  we  also  note  that  the 
policy  case  impacts  relative  to  the  baseline  are  the  same  in  both  simulation  and  static 
prediction  modes. 

The  fourth  step  is  making  a  decision  based  on  the  results  provided  and  the 
preferences  of  the  school  proponent.  This  involves  the  evaluation  by  the  school 
proponent  of  the  tradeoff  between  increased  eligibility  and  reduced  passing  probabilities 
(and  vice-versa). 


Table  R-3. 

Percent  of  simulated  Soldiers  that  meet  or  exceed 
given  chance  of  passing 


Chance  of 
passing 

ST  =  108 

CO 

II 

1 — 

C O 

ST  =  1 1 8 

95.0% 

25.5% 

31.2% 

38.1% 

90.0% 

33.4% 

39.2% 

44.7% 

85.0% 

39.0% 

44.2% 

48.6% 

80.0% 

42.1% 

47.5% 

52.1% 

75.0% 

45.4% 

50.8% 

55.1% 

70.0% 

49.6% 

53.6% 

58.0% 

65.0% 

52.4% 

55.9% 

60.3% 

60.0% 

54.5% 

57.5% 

62.3% 

55.0% 

57.2% 

59.6% 

64.8% 

50.0% 

58.6% 

62.1% 

67.0% 

The  large  standard  error  for  the  constant  allows  scores  to  be  more  spread  out  and  allows  for  large 
groupings  at  the  highest  and  lowest  values.  And  since  the  probabilities  of  passing  are  calculated  from  the 
ratio:  e^' x>  /  (1  +  XJ),  as  the  scores  calculated  from  the  equation  become  extreme,  the  ratios  are  more 
likely  to  approach  zero  and  one. 
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Distribution  for  PASS  EQUATION  113 


X<=0.01  X<=1 


Distribution  for  PASS  EQUATION  118 


X  <=0.02  X  <=1 


163 


164 


42A  (75B):  Personnel  Administrative  Specialist 


The  final  sample  included  2921  soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  soldiers  who  graduated  (97.7%)  or  failed  for  academic  reasons  (2.3%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (55%,  45%  female),  most  had  a  high  school 
diploma  but  not  more  education  (89%,  8%  some  college,  3%  GED  or  less),  and  the 
greatest  number  were  from  Regular  Army  (47%,  18%  Army  Reserve,  35%  National 
Guard).  The  governing  AA  composite,  Clerical  (CL),  for  this  MOS  has  a  cutoff  score  of 
92;  the  sample  mean  is  103.05  (standard  deviation  =  11.115).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 


Table  1.  75B:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

19.526** 

Log  Likelihood 

618.774 

Nagelkerke  R  Square 

.034 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

.447 

1.563 

.242 

GED 

5.23 

1.688 

.260 

COLLEGE 

-.066 

.936 

.019 

RESERVE 

.215 

1.240 

.432 

GUARD 

.965 

2.625 

8.417** 

GENDER 

.169 

1.184 

.445 

CL 

.029 

1.029 

10.891** 

*  =  p  <  .05  ***  =  p<.001 

**  =  p  <  .005 


165 


outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  a  model  including  the  CL  composite,  education  status  (GED  = 
GED  or  less  than  high  school  diploma;  COLLEGE  =  some  college),  Army  component 
(GUARD  =  National  Guard;  RESERVE  =  Army  Reserve),  and  gender  (GENDER  = 
female)  accounts  for  only  three  percent  of  the  variation  in  the  dependent  variable.  This 
model  is  statistically  significant  (%  =  19.526,  P  =  .003),  but  its  explanatory  power 
appears  to  be  limited  (Nagelkerke  R“  =  .034). 

There  are  statistically  significant  effects  for  the  CL  composite  and  Guard 
membership.  The  model  suggests  that  having  a  higher  CL  composite  score  increases  the 
odds  of  passing.  At  the  mean  CL  score,  an  increase  of  one  point  is  associated  with  an 
increase  of  about  three  percent  in  the  odds  of  passing  the  course,  and  a  five-point  increase 
in  CL  would  increase  the  odds  of  passing  by  16%.  There  is  also  a  noteworthy  effect  for 
GUARD:  Guardsmen  have  162%  greater  odds  of  passing  than  Regular  Anny  Soldiers  do. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

Q(P’x>  /  (1  + 

Because  GED,  COLLEGE,  GENDER,  and  RESERVE  were  not  statistically  significant, 
this  analysis  is  confined  to  the  modal  demographics:  male  Soldiers  from  the  Regular 
Army  with  a  high  school  education  (though  results  would  be  similar  for  any  demographic 
combination,  except  Guardsmen).  Based  on  the  model,  when  the  cutoff  score  is  at  its 
current  level  (CL  =  92),  male  Soldiers  with  a  high  school  diploma  and  an  average  CL 
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score  (CL  =  109.62)  have  approximately  a  97%  chance  of  passing.  Currently,  about  92% 
of  Soldiers  are  eligible  for  MOS  75B  assignment  at  the  current  cutoff  (CL  =  92). 
Lowering  the  cutoff  by  five  points  (CL  =  87)  would  increase  eligibility  by  7  percentage 
points  (to  99%)  and  the  average  Soldier  who  would  qualify  for  training  would  have  the 
same  chance  of  passing  (97%).  Raising  the  cutoff  score  by  five  points  (CL  =  97),  the 
average  male  Soldier  with  a  high  school  diploma  would  have  the  same  chance  of  passing 


Table  2.  75B:  Probability  that  a  Soldier  (from  the  larger  Army  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model _ _ _ 


Cutoff  = 

Cutoff = 

Cutoff = 

Cutoff = 

Cutoff = 

82 

87 

92 

97 

102 

Percent  Eligible 
(Regular  Army) 

99.9% 

99.0% 

91.6% 

80.0% 

67.6% 

Mean 

107.50 

107.85 

109.62 

111.79 

114.18 

Passing  rate: 

Male/High  School 

97.2% 

97.3% 

97.4% 

97.6% 

97.7% 

but  eligibility  would  fall  by  about  1 1  percentage  points. 

Figure  1  shows  the  relationship  between  CL  and  the  probability  of  passing  for 
Regular  Anny  male  Soldiers  with  a  high  school  diploma,  including  upper  and  lower 
bounds  based  upon  the  standard  error  of  the  estimated  CL  coefficient.  For  a  particular  CL 
score,  trace  a  vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to 
determine  the  corresponding  probability  of  passing.  For  example,  a  CL  score  of  100 
corresponds  to  a  passing  probability  of  about  92%  for  a  high  school  educated,  Regular 
Army  male  Soldier.  One  can  see  that  even  at  the  lowest  possible  CL  scores  there  is  a  very 
high  chance  of  passing  for  a  given  Soldier.  There  is  a  gradual  increase  in  the  chance  of 
passing  as  CL  increases,  and  the  chance  is  greater  than  95%  at  the  current  cutoff  score 
(CL  =  95). 
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Chance  of  Passing 


Figure  1:  Predicted  Probalility  of  Passing: 
Regular  Army  Male  Soldiers  with  a  High  School  Diploma 


CL  Composite  Score 


1  Mean  —  —  Upper  —  ~  Lower 
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Appendix:  Soldier  Characteristics  (75B) 


Gender 

Male 

Female 

Outcome 

fail 

2.4% 

2.2% 

pass 

97.6% 

97.8% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

pass 

1.1% 

98.9% 

2.3% 

97.7% 

2.3% 

97.7% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

pass 

3.0% 

97.0% 

2.5% 

97.5% 

1 .2% 

98.8% 
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42A  (75H):  Personnel  Services  Specialist 


The  final  sample  included  3024  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (98.7%)  or  failed  for  academic  reasons  (1.3%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  this  sample,  Soldiers  were  split  (50%  male,  50%  female),  most  had  a  high 
school  diploma  but  not  more  education  (89%,  9%  some  college,  2%  GED  or  less),  and 
the  greatest  number  were  from  Regular  Army  (42%,  24%  National  Guard,  34%  Army 
Reserve).  The  governing  AA  composite,  Clerical  (CL),  for  this  MOS  has  a  cutoff  score  of 
92;  the  sample  mean  is  103.05  (standard  deviation  =  1 1.046).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 


Table  1.  75H:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

22.660*** 

Log  Likelihood 

385.494 

Nagelkerke  R  Square 

.059 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

.031 

1.031 

.001 

GUARD 

.086 

1.090 

.035 

RESERVE 

-.234 

.791 

.400 

GENDER 

.817 

2.264 

5.263* 

CL 

.040 

1.041 

20.349*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .005 
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Table  1  and  indicate  that  a  model  including  the  CL  composite,  component  membership 
(GUARD,  RESERVE)  and  gender  (GENDER  =  female)  accounts  for  about  six  percent  of 
the  variation  in  the  dependent  variable.  This  model  is  statistically  significant  (%  =  22.660, 
P  <  .001),  but  its  explanatory  power  is  limited  (Nagelkerke  R  =  .059). 

There  is  a  statistically  significant  effect  for  the  CL  composite.  The  model  suggests 
that  having  a  higher  CL  composite  score  increases  the  odds  of  passing.  At  the  mean  CL 
score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  4%  in  the  odds  of 
passing  the  course,  and  a  five-point  increase  in  CL  would  increase  the  odds  of  passing  by 
22%.  There  is  also  a  significant  effect  for  GENDER,  where  women  have  126%  higher 
odds  of  passing  when  compared  to  men. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  on  the  first  attempt  based  upon 
the  binary  logistic  model.  These  were  calculated  using  the  formula  for  finding  probability 
in  a  binary  logistic  model,  which  is 

qw'x)  /  (1  +  q(P’x>). 

Based  on  the  model,  when  the  cutoff  score  is  at  its  current  level  (CL  =  92),  male  Soldiers 
with  an  average  CL  score  (CL  =  109.62)  have  approximately  a  98.8%  chance  of  passing, 
while  female  Soldiers  have  a  99.5%  chance  of  passing.  Currently,  about  92%  of  Soldiers 
are  eligible  for  MOS  75H  assignment  at  the  current  cutoff  (CL  =  92).  Lowering  the 
cutoff  by  five  points  (CL  =  87)  would  increase  eligibility  by  seven  percentage  points  (to 
99%)  and  the  average  Soldier  who  would  qualify  for  training  would  have  the  same 
chance  of  passing  (male  =  98.7%;  female  =  99.4%).  Raising  the  cutoff  score  by  five 
points  (CL  =  97),  the  average  Soldier  would  have  the  about  same  chance  of  passing  (male 
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=  98.9%;  female  =  99.5%),  but  eligibility  would  fall  by  about  12  percentage  points 
(80%). 


Table  2.  75H:  Probability  that  a  Soldier  (from  the  larger  Army  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model 

Cutoff  = 

82 

Cutoff = 

87 

Cutoff = 

92 

Cutoff = 

97 

Cutoff = 
102 

Percent  Eligible 
(Regular  Army) 

99.9% 

99.0% 

91.6% 

80.0% 

67.6% 

Mean 

107.50 

107.85 

109.62 

111.79 

114.18 

Passing  rates: 

High  School  Male 

98.7% 

98.7% 

99.8% 

98.9% 

99.0% 

High  School  Female 

99.4% 

99.4% 

99.5% 

99.5% 

99.6% 

Figure  1  shows  the  relationship  between  CL  and  the  probability  of  passing  for 
male  and  female  Soldiers,  including  upper  and  lower  bounds  based  upon  the  standard 
error  of  the  estimated  CL  coefficient.  For  a  particular  CL  score,  trace  a  vertical  line  up  to 
the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the  corresponding 
probability  of  passing.  For  example,  a  CL  score  of  100  corresponds  to  a  passing 
probability  of  about  99%  for  a  high  school  educated,  Regular  Army  male  Soldier.  One 
can  see  that  Soldiers  with  a  low  CL  score  have  a  very  high  chance  of  passing  the  course. 
Thus,  as  CL  scores  increase,  there  is  little  room  for  an  increase  in  the  chance  of  passing. 
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Appendix:  Soldier  Characteristics  (75H) 


Gender  1 

Outcome 

Male 

Female  [ 

fail 

1.7% 

.8% 

pass 

98.3% 

99.2% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

.0% 

1 .4% 

.4% 

100.0% 

98.6% 

99.6% 

I  Component  [ 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

1.2% 

1 .6% 

pass 

98.8% 

98.4% 
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42L  (71L):  Administrative  Specialist 


The  final  sample  included  7525  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (99.5%)  or  failed  for  academic  reasons  (0.5%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

Soldiers  were  split  (51%  male,  49%  female),  most  had  a  high  school  diploma  but 
not  more  education  (88%,  8%  some  college,  4%  GED  or  less),  and  were  approximately 
evenly  divided  among  components  (36%  Regular  Army,  29%  National  Guard,  35% 

Army  Reserve).  The  governing  AA  composite,  Clerical  (CL),  for  this  MOS  has  a  cutoff 
score  of  92;  the  sample  mean  is  103.49  (standard  deviation  =  1 1 .020).  See  the  Appendix 
for  a  description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 


Table  1.  71L:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

16.714** 

Log  Likelihood 

439.771 

Nagelkerke  R  Square 

.038 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

.319 

1.376 

.072 

GUARD 

.592 

1.807 

1.704 

RESERVE 

.218 

1.244 

.330 

GENDER 

.806 

2.238 

5.120* 

CL 

.044 

1.045 

14.020*** 

*  =  p  <  .05  ***  =  p<.001 

**  =  p  <  .005 
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Table  1  and  indicate  that  a  model,  including  the  CL  composite,  GUARD,  RESERVE  and 
GENDER,  accounts  for  about  four  percent  of  the  variation  in  the  dependent  variable. 

This  model  is  statistically  significant  (%2  =  16.714,  P  =  .002),  but  its  explanatory  power  is 
limited  (Nagelkerke  R2  =  .038). 

There  is  a  statistically  significant  effect  for  the  CL  composite.  The  model  suggests 
that  having  a  higher  CL  composite  score  increases  the  odds  of  passing.  At  the  mean  CL 
score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  5%  in  the  odds  of 
passing  the  course,  and  a  five-point  increase  in  CL  would  increase  the  odds  of  passing  by 
25%.  There  is  also  a  significant  effect  for  GENDER  where  women  have  124%  higher 
odds  of  passing  when  compared  to  men. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  on  the  first  attempt  based  upon 
the  binary  logistic  model.  These  were  calculated  using  the  formula  for  finding  probability 
in  a  binary  logistic  model,  which  is 

qw'x)  /  (1  +  q(P’x>). 


Table  2.  71L:  Probability  that  a  Soldier  (from  the  larger  Anny  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model 

Cutoff  = 

82 

Cutoff = 

87 

Cutoff = 

92 

Cutoff = 

97 

Cutoff = 
102 

Percent  Eligible 
(Regular  Army) 

99.9% 

99.0% 

91.6% 

80.0% 

67.6% 

Mean 

107.50 

107.85 

109.62 

111.79 

114.18 

Passing  rates: 

Male 

99.4% 

99.4% 

99.4% 

99.5% 

99.5% 

Female 

99.7% 

99.7% 

99.7% 

99.8% 

99.8% 

176 


Based  on  the  model,  when  the  cutoff  score  is  at  its  current  level  (CL  =  92),  male  Soldiers 
with  an  average  CL  score  (CL  =  109.62)  have  approximately  a  99.4%  chance  of  passing, 
while  female  Soldiers  have  a  99.7%  chance  of  passing.  Currently,  about  92%  of  Soldiers 
are  eligible  for  MOS  71L  assignment  at  the  current  cutoff  (CL  =  92).  Lowering  the 
cutoff  by  five  points  (CL  =  87)  would  increase  eligibility  by  seven  percentage  points  (to 
99%),  and  the  average  Soldier  who  would  qualify  for  training  would  practically  have  the 
same  chance  of  passing.  Raising  the  cutoff  score  by  five  points  (CL  =  97)  would  leave 
the  average  Soldier  with  the  same  chance  of  passing,  but  eligibility  would  fall  by  about 
12  percentage  points  (80%). 

Figure  1  shows  the  relationship  between  CL  and  the  probability  of  passing  on  the 
first  attempt  for  male  and  female  Soldiers,  including  upper  and  lower  bounds  based  upon 
the  standard  error  of  the  estimated  CL  coefficient.  For  a  particular  CL  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
corresponding  probability  of  passing.  For  example,  a  CL  score  of  100  corresponds  to  a 
passing  probability  of  about  99%  for  a  high  school  educated,  Regular  Army  male  Soldier. 
One  can  see  that  Soldiers  with  a  relatively  low  CL  score  have  a  very  high  chance  of 
passing  the  course.  Thus,  as  CL  scores  increase,  there  is  little  room  for  an  increase  in  the 
chance  of  passing. 
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Chance  of  Passing 


Figure  1 :  Predicted  Probalility  of  Passing:  Male  Soldiers 


CL  Composite  Scores 


““  Male  ■  -  -  Male  Upper  -  -  -  Male  Lower  Female  —  -  Female  Upper  —  -  Female  Upper 
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Appendix:  Soldier  Characteristics  (71L) 


Gender  I 

Outcome 

Male 

Female  | 

fail 

.6% 

.3% 

pass 

99.4% 

99.7% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

.0% 

.5% 

.2% 

100.0% 

99.5% 

99.8% 

I  Component  | 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

.6% 

.5% 

.3% 

pass 

99.4% 

99.5% 

99.7% 
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44C  (73C):  Finance  Specialist 


The  final  sample  included  1518  soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  soldiers  who  graduated  (92.6%)  or  failed  for  academic  reasons  (7.4%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (56%,  44%  female),  most  had  a  high  school 
diploma  but  not  more  education  (84%,  14%  some  college,  3%  GED  or  less),  and  most 
were  Regular  Army  (53%,  23%  Army  Reserve,  24%  National  Guard).  The  governing  AA 
composite,  Clerical  (CL),  for  this  MOS  has  a  cutoff  score  of  92;  the  sample  mean  is 
105.83  (standard  deviation  =  1 1.141).  See  the  Appendix  for  a  description  of  Soldier 
characteristics  for  this  MOS  training  sample. 


Table  1.  73  C:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

77.037*** 

Log  Likelihood 

722.378 

Nagelkerke  R  Square 

.121 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-7.451 

.001 

29.791*** 

GED 

-8.38 

.433 

3.095 

COLLEGE 

.360 

1.434 

.769 

RESERVE 

.313 

1.368 

1.662 

GUARD 

.366 

1.442 

1.751 

GENDER 

.486 

1.626 

5.483* 

CL 

.094 

1.099 

48.865*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 


A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
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Table  1  and  indicate  that  the  model,  including  the  CL  composite,  education  status  (GED 
=  GED  or  less  than  high  school  diploma;  COLLEGE  =  some  college),  Army  component 
(GUARD  =  National  Guard;  RESERVE  =  Army  Reserve)  and  gender  (GENDER  = 
female),  accounts  for  about  12%  of  the  variation  in  the  dependent  variable.  This  model  is 
statistically  significant  (y2  =  77.037,  P  <  .001),  with  moderate  explanatory  power 
(Nagelkerke  R2  =  .121). 

There  are  statistically  significant  effects  for  the  CL  composite  and  GENDER. 

The  model  suggests  that  having  a  higher  CL  composite  score  increases  the  odds  of 
passing.  At  the  mean  CL  score,  an  increase  of  one  point  is  associated  with  an  increase  of 
about  ten  percent  in  the  odds  of  passing  the  course,  and  a  five-point  increase  in  CL  would 
increase  the  odds  of  passing  by  60%.  There  is  a  noteworthy  effect  for  GENDER,  where 
the  odds  of  a  female  Soldier  passing  this  training  exceed  that  of  the  average  male  Soldier 
by  about  63%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  fonnula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

qw'x)  /  (1  +  q(P’x>). 

Because  GED  and  COLLEGE  were  not  statistically  significant,  this  analysis  is  confined 
to  the  modal  demographics  factored  by  GENDER — high  school  educated  Soldiers  from 
the  Regular  Army  (though  results  would  be  similar  for  any  demographic  combination). 
Based  on  the  model,  when  the  cutoff  score  is  at  its  current  level  (CL  =  92),  male  soldiers 
with  a  high  school  diploma  and  an  average  CL  score  (CL  =  108.87)  have  approximately  a 
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94%  chance  of  passing,  while  the  female  counterparts  have  approximately  a  96%  chance 
of  passing.  Currently,  about  92%  of  soldiers  are  eligible  for  MOS  73C  assignment  at  the 
current  cutoff  (CL  =  92).  Lowering  the  cutoff  by  five  points  (CL  =  87)  would  increase 
eligibility  by  seven  percentage  points,  while  the  average  Soldier  who  would  qualify  for 
training  would  have  a  about  the  same  chance  of  passing  (male  =  93%,  female  =  96%). 
Raising  the  cutoff  score  by  five  points  (CL  =  97),  the  average  Soldier  who  would  still 
qualify  for  the  MOS  would  have  a  about  the  same  chance  of  passing  (male  =  95%,  female 
=  97%),  but  considerably  fewer  Soldiers  would  be  eligible  (79%). 


Table  2.  73C:  Probability  that  a  Soldier  (from  the  larger  Anny  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model _ _ _ 


Cutoff  = 

Cutoff = 

Cutoffs 

Cutoff = 

Cutoff = 

82 

87 

92 

97 

102 

Percent  Eligible 
(Regular  Army) 

99.9% 

99.0% 

91.8% 

79.2%> 

67.3%o 

Mean 

107.26 

107.48 

108.87 

111.22 

113.36 

Passing  rates: 

High  school  male 

93.1% 

93.4% 

94.2% 

95.3% 

96.1% 

High  school  female 

95.7% 

95.8% 

96.3% 

97.0% 

97.6% 

Figure  1  shows  the  relationship  between  CL  and  the  probability  of  passing  for 
Regular  Army  Soldiers,  including  upper  and  lower  bounds  based  upon  the  standard  error 
of  the  estimated  CL  coefficient.  For  a  particular  CL  score,  trace  a  vertical  line  up  to  the 
curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the  corresponding 
probability  of  passing.  For  example,  a  CL  value  of  100  corresponds  to  a  passing 
probability  of  about  88%  for  a  high  school  educated  male  Soldier.  One  can  see  that  the 
probability  of  passing  increases  steeply  over  the  full  range  of  the  data  for  Soldiers  of  both 
gender. 
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Appendix:  Soldier  Characteristics  (73C) 


Gender 

Outcome 

Male 

Female 

fail 

7.6% 

7.1% 

pass 

92.45 

96.7% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

15.8% 

7.8% 

3.3% 

pass 

84.2% 

92.2% 

96.7% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

7.8% 

5.7% 

8.2% 

pass 

92.3% 

94.3% 

91.8% 
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46Q:  Journalist 


The  final  sample  included  402  soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  soldiers  who  graduated  (83.6%)  or  failed  for  academic  reasons  (16.4%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (61%,  39%  female),  most  had  a  high  school 
diploma  but  not  more  education  (64%,  32%  some  college,  4%  GED  or  less),  and  most 
were  Regular  Army  (54%,  33%  Army  Reserve,  13%  National  Guard).  The  governing  AA 
composite,  General  Technical  (GT),  for  this  MOS  has  a  cutoff  score  of  1 10;  the  sample 
mean  is  1 18.65  (standard  deviation  =  10.096).  See  the  Appendix  for  a  description  of 
Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 


Table  1.  46Q:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

40.299*** 

Log  Likelihood 

318.716 

Nagelkerke  R  Square 

.162 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-5.653 

.004 

8.395** 

GED 

-.932 

.384 

1.868 

COLLEGE 

1.103 

3.012 

8.361** 

RESERVE 

.307 

1.360 

1.005 

GUARD 

2.591 

13.338 

7.510* 

GENDER 

.263 

1.266 

.629 

GT 

.057 

1.058 

11.911** 

*  =  p  <  .05  ***  =  p<.001 

**  =  p  <  .005 
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Table  1  and  indicate  that  the  model,  including  the  GT  composite,  education  status  (GED 
=  GED  or  less  than  high  school  diploma;  COLLEGE  =  some  college),  Army  component 
(GUARD  =  National  Guard;  RESERVE  =  Army  Reserve)  and  gender  (GENDER  = 
female),  accounts  for  about  16%  of  the  variation  in  the  dependent  variable.  This  model  is 
statistically  significant  (y2  =  40.299,  P  <  .001),  with  moderate  explanatory  power 
(Nagelkerke  R2  =  .162). 

There  are  statistically  significant  effects  for  the  GT  composite,  having  some 
college  education,  and  National  Guard  membership.  The  model  suggests  that  having  a 
higher  GT  composite  score  increases  the  odds  of  passing.  At  the  mean  GT  score,  an 
increase  of  one  point  is  associated  with  an  increase  of  about  six  percent  in  the  odds  of 
passing  the  course,  and  a  five-point  increase  in  GT  would  increase  the  odds  of  passing  by 
33%.  There  is  also  a  noteworthy  effect  for  COLLGE,  where  having  a  college  education 
increases  the  odds  of  passing  by  200%.  This  finding  is  reasonable  considering  that  those 
with  a  college  education  probably  have  had  more  practice  writing.  Finally,  there  is  a 
noteworthy  effect  for  GUARD,  where  the  odds  of  a  Guardsman  passing  this  training 
exceed  that  of  the  average  Soldier  by  over  1200%.  This  likely  comes  about  because  of 
the  53  Guard  members  who  trained  in  this  MOS,  only  three  failed,  producing  a  large 
effect.29 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  fonnula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

29  To  investigate  the  reliability  of  this  effect,  we  also  estimated  a  model  excluding  the  GUARD  variable, 
and  found  a  relatively  small  impact  on  the  governing  composite.  The  GT  odds  ratio  (factor)  was  reduced 
from  1.058  to  1.050. 
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Q(P’X)  /  (1  +  Q(P’x)y 

Because  GENDER  and  GED  were  not  statistically  significant,  this  analysis  is  confined  to 
the  modal  demographics  factored  by  COLLEGE — male  Soldiers  from  the  Regular  Army 
(though  results  would  be  similar  for  any  demographic  combination,  except  Guardsmen). 
Based  on  the  model,  when  the  cutoff  score  is  at  its  current  level  (GT  =  110),  soldiers  with 
a  high  school  diploma  and  an  average  GT  score  (GT  =  1 18.25)  have  approximately  a 
75%  chance  of  passing,  while  their  compatriots  with  some  college  education  have 
approximately  a  90%  chance  of  passing.  Currently,  about  only  40%  of  soldiers  are 


Table  2.  46Q:  Probability  that  a  Soldier  (from  the  larger  Army  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model 

Cutoff = 
100 

Cutoff = 
105 

Cutoff = 
110 

Cutoff = 
115 

Cutoff = 
120 

Percent  Eligible 
(Regular  Army) 

69.9% 

53.3% 

39.7% 

24.6% 

15.1% 

Mean 

112.42 

115.60 

118.25 

121.71 

124.51 

Passing  rates: 

High  School  &  GED 
male 

68.0% 

71.8% 

74.8% 

78.3% 

80.9% 

College  male 

86.5% 

88.4% 

89.9% 

91.6% 

92.7% 

eligible  for  MOS  46Q  assignment  at  the  current  cutoff  (GT  =  110).  Lowering  the  cutoff 
by  five  points  (GT  =  105)  would  increase  eligibility  by  13  percentage  points,  while  the 
average  male  high  school  educated  Soldier  who  would  qualify  for  training  would  have  a 
modestly  lower  chance  of  passing.  Raising  the  cutoff  score  by  five  points  (GT  =  115), 
the  average  Soldier  who  would  still  qualify  for  the  MOS  would  have  a  modestly  higher 
chance  of  passing,  but  many  fewer  Soldiers  would  be  eligible  (25%). 

Figure  1  shows  the  relationship  between  GT  and  the  probability  of  passing  for 
Regular  Army  Soldiers,  including  upper  and  lower  bounds  based  upon  the  standard  error 
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of  the  estimated  GT  coefficient.  For  a  particular  GT  score,  trace  a  vertical  line  up  to  the 
curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the  corresponding 
probability  of  passing.  For  example,  a  GT  value  of  100  corresponds  to  a  passing 
probability  of  about  52%  for  a  high  school  educated  male  Soldier.  One  can  see  that  the 
probability  of  passing  increases  steeply  over  the  full  range  of  the  data  for  Soldiers  in  both 
education  categories.  However,  the  increase  among  those  with  some  college  rises  much 
earlier  than  those  who  have  a  high  school  diploma  or  less. 


Figure  1:  Predicted  Probability  of  Passing:  Male,  Regular  Army 


GT  composite  score 


I^^^High  School  ■  -  -  HS  Upper  -  ■  ■  HS  Lower  ~~  College  —  ■  College  Upper  — 1  ~  College  Lower 
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Appendix:  Soldier  Characteristics  (46Q) 


Gender 

Male 

Female 

Outcome 

fail 

17.8% 

14.2% 

pass 

82.2% 

85.8% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

pass 

28.6% 

71.4% 

20.0% 

80.0% 

7.8% 

92.2% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

pass 

19.0% 

81.0% 

16.5% 

83.5% 

5.7% 

94.3% 
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46Q  Risk  Analysis  of  Policy  Alternatives 


In  this  section  we  extend  the  policy  analysis  with  a  risk  analysis  of  the  policy 
alternatives  considered  using  simulation  methods.  We  employ  the  “@RISK”  software 
package  which  is  designed  as  an  add-in  for  Microsoft  Excel.  There  are  four  steps  in  the 
risk  analysis. 

The  first  step  is  developing  a  model.  A  regression  model  for  predicting  the 
passing  rate  has  been  specified  and  estimated,  and  the  results  stored  in  Excel  worksheet 
format.  See  Table  1. 

The  second  step  is  identifying  the  uncertainty  inherent  in  both  the  input  variables 
(composite  scores,  Soldier  characteristics)  and  the  corresponding  estimated  parameters 
and  specifying  their  possible  values  with  probability  distributions. 

To  approximate  the  governing  composite  distribution,  we  use  the  larger  Army 
contract  population  (EAF  file:  those  who  contracted  from  Jan  1992  to  Aug  2003).  This 
distribution  can  be  assumed  to  follow  a  normal  distribution.  The  mean  and  standard 
deviation  estimated  from  the  contract  population  and  used  as  input  to  the  simulations  are 
shown  in  Table  R-l  (before  truncation). 


Table  R-l.  Input  infonnation  for  the  simulations 

Mean 

Std  error  / 
deviation 

Constant 

-5.653 

1.951 

GT  var 

106.23  (before  truncation) 

11.149 

GT  coeff 

.057 

.016 

GED  var 

3.5% 

GED  coeff 

-.932 

.682 

COLLEGE  var 

31.8% 

COLLEGE  coeff 

1.103 

.381 

GUARD  var 

13.2% 

GUARD  coeff 

2.591 

.945 

RESERVE  var 

33.1% 

RESERVE  coeff 

.307 

.306 

GENDER  var 

38.6% 

GENDER  coeff 

.236 

.297 

30  Software  is  available  from  Palisade  Corporation,  Newfield,  NY. 
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To  approximate  the  distributions  for  the  demographic  variables  we  rely  on  the 
training  data  sample  used  to  estimate  the  regression  model.  These  distributions  can  be 
assumed  to  follow  binomial  distributions  with  means  estimated  from  the  training  data 
sample  as  shown  in  Table  R-l. 

To  approximate  the  distributions  for  the  estimated  parameters  in  the  logit  model, 
we  note  that,  under  very  general  conditions,  the  estimates  will  be  asymptotically  normally 
distributed.31  The  estimated  coefficients  (means  and  standard  errors)  are  shown  in  Table 
R-l. 


The  third  step  is  analyzing  the  model  with  simulation  to  determine  the  range  and 
probabilities  of  all  possible  training  outcome  results.  The  results  reported  represent  the 

T9 

simulation  of  the  regression  prediction  equation  for  1000  iterations  using  a  fixed  seed. 

The  comparisons  are  between  the  baseline  case  (GT  =110)  and  the  policy  cases  (GT  = 

105  and  GT  =  115).  These  cases  are  delineated  by  the  cutoff  score  level  which  serves  as 
the  lower  truncation  point  in  the  governing  composite  input  distribution.  The  simulation 
results  in  Table  R-2  describe  the  effect  on  the  mean  probability  of  passing  as  the  GT 
cutoff  score  level  is  varied,  and  represent  an  improvement  of  the  policy  analysis 
presented  in  Table  2  because  they  reflect  the  simultaneous  effects  of  all  known  sources  of 
uncertainty. 


Table  R-2.  46Q  simulation  resu' 

Its 

GT  =  105 

O 

H 

II 

o 

GT=  115 

Mean  governing 
composite 

114.36 

117.65 

121.36 

Mean  probability 
of  passing 

68.9% 

71.4% 

72.6% 

Std  deviation 

33.5% 

32.1% 

33.5% 

The  frequency  distributions  of  simulated  effects  are  shown  below  in  the  histogram 
graphs.  The  histogram  portrays  the  full  range  of  outcomes  that  average  to  the  mean 
effect  as  reported.  Looking  at  the  histogram  for  the  baseline  case  (GT  =  110),  probability 
of  passing  is  plotted  on  the  horizontal  axis  and  the  height  of  each  bar  indicates  the 
relative  number  of  Soldiers  at  that  passing  probability.  The  information  in  the  histograms 
is  effectively  summarized  in  Table  R-3.  In  that  table  we  see  the  percentage  of  Soldiers 


31  The  intuition  is  that  logit  estimates  are  derived  from  maximum  likelihood  (ML)  estimation,  and  under 
fairly  general  conditions  ML  estimates  are  consistent  and  asymptotically  normally  distributed. 

32  Latin  Hypercube  (LH)  sampling  is  utilized  in  drawing  samples  for  the  input  distributions.  It  is  designed 
to  accurately  recreate  the  input  distribution  through  sampling  in  fewer  iterations  when  compared  with  the 
Monte  Carlo  method.  The  key  to  LH  sampling  is  stratification  of  the  input  probability  distributions. 
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who  meet  or  exceed  a  given  chance  of  passing.  For  example,  looking  at  the  baseline  case 
(GT  =  110),  about  65%  of  Soldiers  have  a  70%  or  greater  chance  of  passing. 


Table  R-3. 

Percent  of  simulated  Soldiers  that  meet  or  exceed 
given  chance  of  passing 


Chance  of 
passing 

GT  =  105 

CD 

H 

ll 

o 

GT  =  115 

95.0% 

33.8% 

35.0% 

37.9% 

90.0% 

43.3% 

44.5% 

48.2% 

85.0% 

49.0% 

50.3% 

54.0% 

80.0% 

53.8% 

56.0% 

59.0% 

75.0% 

58.6% 

61.2% 

63.7% 

70.0% 

61.7% 

65.1% 

66.6% 

65.0% 

64.0% 

68.4% 

68.8% 

60.0% 

66.8% 

71.1% 

71.1% 

55.0% 

69.0% 

73.4% 

73.7% 

50.0% 

71.2% 

75.6% 

75.7% 

We  find  that  the  mean  probability  of  passing  in  the  simulation  is  71%  (Table  R-2, 
baseline).  Looking  at  the  histograms,  we  see  a  large  portion  of  Soldiers  in  the  highest 
range  of  passing  scores,  and  the  remainder  distributed  more  or  less  uniformly  across  the 
entire  range.  We  note  that  the  mean  simulated  passing  probability  turns  out  to  be  slightly 
lower  than  the  static  prediction  at  the  mean  GT  score  (Table  2).  In  the  simulated 
predictions  we  are  calculating  an  average  over  all  Soldiers  and  taking  into  account  all 
sources  of  uncertainty,  whereas  in  the  static  prediction  we  are  calculating  the  probability 
of  passing  only  at  the  mean  GT  score,  ignoring  the  distribution  of  Soldier  scores  etc.  as 
well  as  uncertainty.  While  recognizing  this  variability  is  important,  we  also  note  that  the 
policy  case  impacts  relative  to  the  baseline  are  the  same  in  both  simulation  and  static 
prediction  modes. 

The  fourth  step  is  making  a  decision  based  on  the  results  provided  and  the 
preferences  of  the  school  proponent.  This  involves  the  evaluation  by  the  school 
proponent  of  the  tradeoff  between  increased  eligibility  and  reduced  passing  probabilities 
(and  vice-versa). 
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Distribution  for  PASS  EQUATION  110 


X  <=0.05  X  <=1 


Distribution  for  PASS  EQUATION  115 


X  <=0.06  X  <=1 
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Distribution  for  PASS  EQUATION  105 


X  <=0.04  X<=1 
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46R:  Broadcast  Journalist 


The  final  sample  included  259  soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  soldiers  who  graduated  (74.5%)  or  failed  for  academic  reasons  (25.5%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (58%,  42%  female),  most  had  a  high  school 
diploma  but  not  more  education  (68%,  29%  some  college,  3%  GED  or  less),  and  most 
were  Regular  Army  (69%,  26%  Army  Reserve,  15%  National  Guard).  The  governing  AA 
composite,  General  Technical  (GT),  for  this  MOS  has  a  cutoff  score  of  1 10;  the  sample 
mean  is  1 17.56  (standard  deviation  =  6.446).  See  the  Appendix  for  a  description  of 
Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 


Table  1.  46R:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

30.654*** 

Log  Likelihood 

236.350 

Nagelkerke  R  Square 

.164 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-10.399 

.000 

10.633** 

GED 

1.593 

4.920 

1.657 

COLLEGE 

1.013 

2.753 

6.241* 

RESERVE 

-.102 

.903 

.087 

GUARD 

.663 

1.941 

1.824 

GENDER 

.864 

2.373 

6.948** 

GT 

.093 

1.097 

11.712** 

*  =  p  <  .05  ***  =  p<.001 

**  =  p  <  .005 
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Table  1  and  indicate  that  a  model  including  the  GT  composite,  education  status  (GED  = 
GED  or  less  than  high  school  diploma;  COLLEGE  =  some  college),  Army  component 
(GUARD  =  National  Guard;  RESERVE  =  Army  Reserve),  and  gender  (GENDER  = 
female)  accounts  for  about  16%  of  the  variation  in  the  dependent  variable.  This  model  is 
statistically  significant  (y2  =  30.654,  P  <  .001),  with  moderate  explanatory  power 
(Nagelkerke  R2  =  .164). 

There  are  statistically  significant  effects  for  the  GT  composite,  COLLEGE,  and 
GENDER.  The  model  suggests  that  having  a  higher  GT  composite  score  increases  the 
odds  of  passing.  At  the  mean  GT  score,  an  increase  of  one  point  is  associated  with  an 
increase  of  about  10%  in  the  odds  of  passing  the  course,  and  a  five-point  increase  in  GT 
would  increase  the  odds  of  passing  by  60%.  There  is  also  a  noteworthy  effect  for 
COLLGE:  having  a  college  education  increases  the  odds  of  passing  by  175%.  This 
finding  is  reasonable  considering  that  those  with  a  college  education  probably  have  had 
more  practice  with  skills  associated  with  writing  and  public  speaking.  Finally,  there  is  a 
noteworthy  effect  for  GENDER,  where  the  odds  of  a  female  soldier  passing  this  training 
exceed  that  of  the  average  male  soldier  by  a  factor  of  2.37  (or  137%).  The  reason  for  this 
remains  unclear,  but  may  be  associated  with  better  verbal  ability  among  women. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  on  the  first  attempt  based  upon 
the  binary  logistic  model.  These  were  calculated  using  the  formula  for  finding  probability 
in  a  binary  logistic  model,  which  is 

qw'x)  /  (1  +  q(P’x>). 
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Because  GED,  RESERVE,  and  GUARD  were  not  statistically  significant,  this  analysis  is 
confined  to  the  modal  demographics  factored  by  COLLEGE  and  GENDER:  Soldiers 
from  the  Regular  Army  with  a  high  school  or  college  education  (though  results  would  be 
similar  for  any  demographic  combination).  Based  on  the  model,  when  the  cutoff  score  is 
at  its  current  level  (GT  =  110),  male  Soldiers  with  a  high  school  diploma  and  an  average 
GT  score  (GT  =  1 18.25)  have  approximately  a  65%  chance  of  passing,  while  their  female 
counterparts  with  a  high  school  diploma  have  approximately  an  81%  chance  of  passing. 
With  some  college  education,  male  Soldiers  have  almost  a  20  percentage  point  better 
chance  of  passing  while  female  Solders  have  an  1 1  percentage  point  better  chance. 
Currently,  about  only  40%  of  Soldiers  are  eligible  for  MOS  46R  assignment  at  the 
current  cutoff  (GT  =  110).  Raising  the  cutoff  score  by  five  points  (GT  =  115),  the 


Table  2.  46R:  Probability  that  a  Soldier  (from  the  larger  Anny  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model 

Cutoff = 
100 

Cutoff = 
105 

Cutoff = 
110 

Cutoff = 
115 

Cutoff = 
120 

Percent  Eligible 
( Regular  Army) 

69.9% 

53.3% 

39.7% 

24.6% 

15.1% 

Mean 

112.42 

115.60 

118.25 

121.71 

124.51 

Passing  rates: 

Male  High  School 

51.4% 

58.7% 

64.5% 

71.5% 

76.5% 

Male  College 

74.4% 

79.7% 

83.4% 

87.4% 

90.0% 

Female  High  School 

71.5% 

77.1% 

81.2% 

85.6% 

88.5% 

Female  College 

87.3% 

90.3% 

92.2% 

94.3% 

95.5% 

average  qualifying  male  Soldier  with  a  high  school  diploma  would  have  a  noticeably 
higher  chance  of  passing,  but  many  fewer  Soldiers  would  be  eligible  for  training. 
Lowering  the  cutoff  by  five  points  (GT  =  105),  a  much  larger  number  of  Soldiers  would 
be  able  to  qualify  for  training  with  a  modest  adverse  impact  on  passing  rates  for  the 
average  Soldier  (except  for  the  average  high  school  educated,  male  Soldier). 
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Figure  1  shows  the  relationship  between  GT  and  the  probability  of  passing  for 
Regular  Army  Soldiers  factored  by  COLLEGE  and  GENDER.  For  a  particular  GT  score, 
trace  a  vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to 
determine  the  corresponding  probability  of  passing.  For  example,  a  GT  value  of  100 


Figure  1:  Predicted  Probability  of  Passing:  Education  by  Sex 


GT  composite  score 


—  -  Col  Female  ■—  Col  Male  —  —  HS  Female  -  -  -  FIS  Male 


corresponds  to  a  passing  probability  of  about  48%  for  a  male  Soldier  with  some  college. 
One  can  see  that  the  probability  of  passing  increases  steeply  over  the  full  range  of 
possible  GT  scores  for  Soldiers  of  both  genders  in  both  education  categories.  However, 
the  increase  among  female  Soldiers  with  some  college  rises  much  earlier  than  those  who 
have  a  high  school  diploma  or  less.  The  increase  among  male  Soldiers  with  a  high  school 
diploma  or  less  education  rises  last,  and  at  the  current  cutoff  score  (GT  =110)  these 
Soldiers  stand  almost  a  20  percentage  point  smaller  chance  of  passing  the  course. 
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Figure  2  shows  the  curve  for  the  modal  category  male  Soldiers  with  a  high  school 
diploma  with  the  upper  and  lower  bounds  of  GT  scores  based  upon  the  standard  error  of 
the  estimated  GT  coefficient.  The  bounds  are  quite  tight  for  these  soldiers,  except  at  the 
extremes.  Bounds  for  the  Soldiers  from  other  demographic  groups  would  look  the  same. 
This  suggests  that  GT  is  a  consistent  predictor  of  success. 
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Appendix:  Soldier  Characteristics  (46R) 


Gender 

Male 

Female 

Outcome 

fail 

19.3% 

pass 

80.7% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

pass 

12.5% 

87.5% 

31 .6% 

68.4% 

12.2% 

87.8% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

pass 

25.7% 

74.3% 

29.4% 

70.6% 

17.9% 

82.1% 
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44B:  Metal  Worker 


The  final  sample  included  860  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (86.4%)  or  failed  for  academic  reasons  (13.6%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (90%,  10%  female),  most  had  a  high  school 
diploma  but  not  more  education  (89%,  2%  some  college,  9%  GED  or  less),  and  most 
were  from  the  Regular  Army  (43%,  37%  Guard,  20%  Reserve).  The  governing  AA 
composite,  General  Maintenance  (GM),  for  this  MOS  has  a  cutoff  score  of  93;  the  sample 
mean  is  101.74  (standard  deviation  =  12.647).  This  MOS  has  an  alternative  governing 
AA  composite  cutoff,  where  soldiers  with  an  GM  score  of  88  can  also  qualify  if  they 
have  a  General  Technical  (GT)  score  of  85;  the  sample  mean  for  GT  is  100.97  (standard 
deviation  =  1 1.696);  GT  and  GM  are  highly  correlated  (r  =  .787).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  for  the  first 
model  are  reported  in  Table  1  and  indicate  that  the  model,  including  the  GM  composite, 
education  level  (GED  =  GED  or  less  education,  COLLEGE  =  some  college),  Anny 
component  (GUARD  =  National  Guard,  RESERVE  =  Army  Reserve),  and  gender 
(GENDER  =  female),  accounts  for  about  7%  of  the  variation  in  the  dependent  variable. 
This  model  is  statistically  significant  (x  =  31.089,  P  <  .001),  and  has  limited  explanatory 
power  (Nagelkerke  R  =  .065). 
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Table  1.  44B:  Results  of  ti 
prediction  model:  GM  onl 

he  binary  logistic 
y  model 

Chi-  Square 

31.089*** 

Log  Likelihood 

652.991 

Nagelkerke  R  Square 

.065 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-1.894 

.151 

4.158* 

GED 

-.074 

.929 

.040 

COLLEGE 

.638 

1.893 

.374 

GUARD 

.201 

1.222 

.657 

RESERVE 

-6.27 

.534 

6.134* 

GENDER 

.269 

1.309 

.613 

GM 

.038 

1.039 

16.866*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p<  .01 

There  are  statistically  significant  effects  for  the  GM  composite  and  Reserve 
membership.  The  model  suggests  that  having  a  higher  GM  composite  score  increases  the 
odds  of  passing.  At  the  mean  GM  score,  an  increase  of  one  point  is  associated  with  an 
increase  of  about  four  percent  in  the  odds  of  passing  the  course,  and  a  five-point  increase 
in  GM  would  increase  the  odds  of  passing  by  21%.  There  is  also  a  notable  effect  for 
Army  Reserve  membership  where  Reservists  have  47%  lower  odds  of  passing. 

A  second  model,  which  included  the  same  variables  from  the  first  equation  as 
well  as  the  GT  composite,  accounted  for  the  same  amount  of  variation  in  the  dependent 
variable.  The  GT  composite  did  not  have  a  significant  effect.  The  results  are  shown  in 
Appendix  B. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model  that  includes  only  GM.  These  probabilities  were  calculated  using  the  formula  for 
finding  probability  in  a  binary  logistic  model,  which  is 
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Q(P’x)  /  (1  +  Q(P’x)y 

Because  GED,  COLLEGE,  GUARD,  and  GENDER  were  not  statistically  significant,  this 
analysis  is  confined  to  the  modal  demographic — Regular  Army  high  school  educated 
male  Soldiers  (though  results  would  be  similar  for  any  demographic  combination,  except 
Reservists).  Based  on  the  model,  when  the  cutoff  score  is  at  its  current  levels  (GM  =  93; 
or  GM  =  88  &  GT  =  85),  male  Soldiers  with  a  high  school  diploma  and  average  scores 
(GT  =  106.47,  GM  =  107.55)  have  approximately  a  90%  chance  of  passing.  Currently, 
about  86%  of  Soldiers  are  eligible  for  MOS  44B  assignment  at  the  current  cutoff. 
Lowering  the  GM  cutoff  by  five  points  (GM  =  88;  or  GM  =  83  &  GT  =  85)  would 
increase  eligibility  by  nine  percentage  points,  while  the  average  male  Soldier  who  would 
qualify  for  training  would  have  essentially  the  same  chance  of  passing  (89%).  Raising  the 
GT  and  GM  cutoff  scores  by  five  points  each  (GM  =  98;  or  GM  =  93  &  GT  =  90),  the 
average  male  Soldier  who  would  still  qualify  for  the  MOS  would  have  about  the  same 
chance  of  passing  (90%),  but  fewer  Soldiers  would  be  eligible  (71%)  . 


Table  2.  44B:  Probability  that  a  Soldier  (from  the  larger  Anny  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model _ _ _ 


Cutoff: 

Cutoff: 

Cutoff: 

Cutoff: 

Cutoff: 

GM  =  83 

GM  =  88 

GM  =  93 

GM  =  98 

GM  =  103 

-or- 

-or- 

-or- 

-or- 

-or- 

GM  =  78 

GM  =  83 

GM  =  88 

GM  =  93 

GM  =  98 

& 

& 

& 

& 

& 

GT  =  85 

O 

H 

II 

oo 

O 

H 

II 

oo 

O 

H 

II 

'O 

o 

O 

H 

II 

'O 

Percent  Eligible 
(Regular  Army) 

98.6% 

94. 7% 

85.7% 

77.4% 

56.6% 

Mean:  GM 

104.31 

104.99 

106.47 

108.89 

111.51 

Mean:  GT 

106.37 

106.72 

107.55 

109.18 

111.07 

Passing  rates: 

High  school  male 

88.8% 

89.0% 

89.6% 

90.4% 

91.2% 
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Figure  1  shows  the  relationship  between  GM  and  the  probability  of  passing  for 
high  school  educated  male  Soldiers,  including  upper  and  lower  bounds  based  upon  the 
standard  error  of  the  estimated  GM  coefficient.  For  a  particular  GM  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
corresponding  probability  of  passing.  For  example,  a  GM  value  of  100  corresponds  to  a 
passing  probability  of  about  87%  for  a  high  school  educated  male  Soldier. 


Figure  1 :  Predicted  Probability  of  Passing:  Regular  Army  High  School  Male 


School  /  Male  —  —  HSM  Upper  —  —  HSM  Lower 


33  Appendix  C  shows  the  percent  of  Soldiers  eligible  under  primary  and  alternative  cutoff  criteria. 
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Appendix  A:  Soldier  Characteristics  (44B) 


Gender 

Outcome 

Male 

Female 

fail 

13.6% 

■B 

pass 

86.4% 

— 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

14.7% 

13.7% 

5.9% 

pass 

85.3% 

86.3% 

94.1% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

12.6% 

21.0% 

mmm 

pass 

87.4% 

79.0% 
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45B:  Small  Arms  Repairer 


The  final  sample  included  540  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (91.7%)  or  failed  for  academic  reasons  (8.3%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (77%,  23%  female),  most  had  a  high  school 
diploma  but  not  more  education  (88%,  6%  some  college,  6%  GED  or  less),  and  most 
were  from  the  Regular  Army  (60%,  29%  Guard,  1 1%  Reserve).  The  governing  AA 
composite,  General  Maintenance  (GM),  for  this  MOS  has  a  cutoff  score  of  93;  the  sample 
mean  is  101.34  (standard  deviation  =  12.051).  This  MOS  has  an  alternative  governing 
AA  composite  cutoff,  where  soldiers  with  a  GM  score  of  88  can  also  qualify  if  they  have 
a  General  Technical  (GT)  score  of  85;  the  sample  mean  for  GT  is  101.74  (standard 
deviation  =  12.05 1);  GT  and  GM  are  highly  correlated  (r  =  .808).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  model,  including  the  GM  composite,  education  level  (GED 
=  GED  or  less  education,  COLLGE  =  some  college),  Army  component  (GUARD  = 
National  Guard,  RESERVE  =  Army  Reserve),  and  gender  (GENDER  =  female), 
accounts  for  about  17%  of  the  variation  in  the  dependent  variable.  This  model  is 
statistically  significant  (%  =  41.351,  P  <  .001),  and  has  moderate  explanatory  power 
(Nagelkerke  R2  =  .169). 
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Table  1.  45B:  Results  of  ti 
prediction  model:  GM  onl 

he  binary  logistic 
y  model 

Chi-  Square 

41.351*** 

Log  Likelihood 

268.432 

Nagelkerke  R  Square 

.169 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-9.272 

.000 

16.918*** 

GED 

.157 

1.170 

.040 

COLLEGE 

-.745 

.475 

1.182 

GUARD 

-.043 

.957 

.013 

RESERVE 

-.397 

.673 

.545 

GENDER 

.434 

1.544 

1.351 

GM 

.121 

1.129 

25.507*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p<  .01 

There  is  a  statistically  significant  effect  for  the  GM  composite.  The  model 
suggests  that  having  a  higher  GM  composite  score  increases  the  odds  of  passing.  At  the 
mean  GM  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  13%  in 
the  odds  of  passing  the  course,  and  a  five-point  increase  in  GM  would  increase  the  odds 
of  passing  by  83%.34 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  probabilities  were  calculated  using  the  formula  for  finding  probability  in  a 
binary  logistic  model,  which  is 

q(P'x)  /  (1  +  q(P’x)). 

34 

A  second  model,  which  included  the  same  variables  from  the  first  equation  as  well  as  the  GT 
composite,  accounted  for  approximately  18%  of  the  variation  in  the  dependent  variable.  This  model  is  also 
statistically  significant  (y2  =  44.946,  P  <  .001).  Regression  model  results  are  reported  in  Appendix  B. 
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Because  GED,  COLLEGE,  GUARD,  RESERVE,  and  GENDER  were  not  statistically 
significant,  this  analysis  is  confined  to  the  modal  demographic — Regular  Army  high 
school  educated  male  Soldiers  (though  results  would  be  similar  for  any  demographic 
combination).  Based  on  the  model,  when  the  cutoff  score  is  at  its  current  levels  (GM  = 

93;  or  GM  =  88  &  GT  =  85),  male  Soldiers  with  a  high  school  diploma  and  an  average 
GM  score  (GM  =  107.55)  have  approximately  a  97%  chance  of  passing.  Currently,  about 
86%  of  Soldiers  are  eligible  for  MOS  45B  assignment  at  the  current  cutoff.  Lowering  the 
GM  cutoff  by  five  points  (GM  =  88;  or  GM  =  83  &  GT  =  85)  would  increase  eligibility 
by  nine  percentage  points,  while  the  average  male  Soldier  who  would  qualify  for  training 
would  have  essentially  the  same  chance  of  passing  (97%).  Raising  the  GT  and  GM  cutoff 
scores  by  five  points  each  (GM  =  98;  or  GM  =  93  &  GT  =  90),  the  average  male  Soldier 
who  would  still  qualify  for  the  MOS  would  have  about  the  same  chance  of  passing 
(98%),  but  fewer  Soldiers  would  be  eligible  (71%).35 


Table  2.  45B:  Probability  that  a  Soldier  (from  the  larger  Anny  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model _ _ _ 


Cutoff: 

Cutoff: 

Cutoff: 

Cutoff: 

Cutoff: 

GM  =  83 

GM  =  88 

GM  =  93 

GM  =  98 

GM  =  103 

-or- 

-or- 

-or- 

-or- 

-or- 

GM  =  78 

GM  =  83 

GM  =  88 

GM  =  93 

GM  =  98 

& 

& 

& 

& 

& 

in 

00 

II 

H 

O 

in 

00 

II 

H 

O 

in 

00 

II 

H 

O 

O 

H 

II 

so 

o 

in 

O' 

II 

H 

O 

Percent  Eligible 
(Regular  Army) 

98.6% 

94. 7% 

85.7% 

77.4% 

56. 6%> 

Mean:  GM 

104.31 

104.99 

106.47 

108.89 

111.51 

Mean:  GT 

106.37 

106.72 

107.55 

109.18 

111.07 

Passing  rates: 

High  School  Male 

96.6% 

96.9% 

97.4% 

98.0% 

98.6% 

35  Appendix  C  shows  the  percent  of  Soldiers  who  are  eligible  according  to  primary  and  alternative  cutoff 
criteria. 
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Figure  1  shows  the  relationship  between  GM  and  the  probability  of  passing  for 
high  school  educated  male  Soldiers,  including  upper  and  lower  bounds  based  upon  the 
standard  error  of  the  estimated  GM  coefficient.  For  a  particular  GM  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
corresponding  probability  of  passing.  For  example,  a  GM  value  of  100  corresponds  to  a 
passing  probability  of  about  95%  for  a  high  school  educated  male  Soldier. 


Figure  1 :  Predicted  Probability  of  Passing:  Regular  Army  High  School  Male 


School  /  Male  —  —  HSM  Upper  —  —  HSM  Lower 
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Appendix  A:  Soldier  Characteristics  (45B) 


Gender 

Outcome 

Male 

Female 

fail 

7.7% 

10.4% 

pass 

92.3% 

89.6% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

5.7% 

8.4% 

9.0% 

pass 

94.3% 

91.6% 

91.0% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

8.0% 

8.3% 

9.0% 

pass 

92.0% 

91.7% 

91.0% 
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Appendix  B 


Table  la.  45B:  Results  of 
prediction  model:  GM  anc 

the  binary  logistic 

1  GT  model 

Chi-  Square 

44  (^46^^^ 

Log  Likelihood 

264.837 

Nagelkerke  R  Square 

.183 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-10.792 

.000 

18.781*** 

GED 

.217 

1.243 

.076 

COLLEGE 

-.819 

.441 

1.398 

GUARD 

-.035 

.966 

.008 

RESERVE 

-.526 

.591 

.939 

GENDER 

.372 

1.451 

.969 

GM 

.089 

1.093 

9.183** 

GT 

.048 

1.049 

3.488 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 
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Appendix  C:  Percent  Eligible 


Table  2a. 

Percent  eligible  under  primary  and  alternative  cutoff 
criteria. 

Cutoff 

Qualify 

Both 

Qualify 

GM 

Only 

Qualify 

GM/GT 

Only 

Qualify 

Total 

Do  Not 
Qualify 

-10 

95.2% 

0.9% 

2.5% 

98.6% 

1.4% 

-5 

87.6% 

0.7% 

6.4% 

94. 7% 

5.3% 

Current 

75.1% 

0.4% 

10.2% 

85.7% 

14.3% 

+5 

59.9% 

1.5% 

10.0% 

71.4% 

28.6% 

+  10 

44.0% 

1.8% 

10.8% 

56.6% 

43.4% 
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52D:  Power  Generation  Equipment  Repairer 


The  final  sample  included  3275  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (97.5%)  or  failed  for  academic  reasons  (2.5%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (87%,  13%  female),  most  had  a  high  school 
diploma  but  not  more  education  (88%,  5%  some  college,  7%  GED  or  less),  and  most 
were  from  the  Regular  Army  (60%,  20%  Guard,  20%  Reserve).  The  governing  AA 
composite,  General  Maintenance  (GM),  for  this  MOS  has  a  cutoff  score  of  97;  the  sample 
mean  is  105.53  (standard  deviation  =  1 1.059).  This  MOS  has  an  alternative  governing 
AA  composite  cutoff,  where  soldiers  with  a  GM  score  of  88  can  also  qualify  if  they  have 
a  General  Technical  (GT)  score  of  85;  the  sample  mean  for  GT  is  106.06  (standard 
deviation  =  10.158);  GT  and  GM  are  highly  correlated  (r  =  .708).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  model,  including  the  GM  composite,  education  level  (GED 
=  GED  or  less  education,  COLLGE  =  some  college),  Army  component  (GUARD  = 
National  Guard,  RESERVE  =  Army  Reserve),  and  gender  (GENDER  =  female), 
accounts  for  about  six  percent  of  the  variation  in  the  dependent  variable.  This  model  is 
statistically  significant  (y2  =  37.860,  P  <  .001),  and  has  somewhat  limited  explanatory 
power  (Nagelkerke  R  =  .055). 
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Table  1.  52D:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

37.860*** 

Log  Likelihood 

736.107 

Nagelkerke  R  Square 

.055 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-2.470 

.085 

4.236* 

GED 

.494 

1.638 

.636 

COLLEGE 

.033 

1.034 

.003 

GUARD 

.120 

1.128 

.165 

RESERVE 

.630 

1.877 

2.911 

GENDER 

.166 

1.180 

.284 

GM 

.058 

1.060 

24.078*** 

*  =  p  <  .05  ***=p<.001 

**  =  p<  .01 


There  is  a  statistically  significant  effect  for  the  GM  composite.  The  model 
suggests  that  having  a  higher  GM  composite  score  increases  the  odds  of  passing.  At  the 
mean  GM  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  six 
percent  in  the  odds  of  passing  the  course,  and  a  six-point  increase  in  GM  would  increase 
the  odds  of  passing  by  33%. 36 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  probabilities  were  calculated  using  the  formula  for  finding  probability  in  a 
binary  logistic  model,  which  is 

q(P'x)  /  (1  +  q(P’x)). 

36 

A  second  model,  which  included  the  same  variables  from  the  first  equation  as  well  as  the  GT 
composite,  accounted  for  approximately  6%  of  the  variation  in  the  dependent  variable.  This  model  is  also 
statistically  significant  ( X  =  38.048,  P  <  .001).  Regression  model  results  are  reported  in  Appendix  B. 


214 


Because  GED,  COLLEGE,  GUARD,  RESERVE,  and  GENDER  were  not  statistically 
significant,  this  analysis  is  confined  to  the  modal  demographic — Regular  Army  high 
school  educated  male  Soldiers  (though  results  would  be  similar  for  any  demographic 
combination).  Based  on  the  model,  when  the  cutoff  score  is  at  its  current  levels  (GM  = 

97;  or  GM  =  88  &  GT  =  85),  male  Soldiers  with  a  high  school  diploma  and  an  average 
GM  score  (GM  =  106.88)  have  approximately  a  98%  chance  of  passing.  Currently,  about 
83%  of  Soldiers  are  eligible  for  MOS  52D  assignment  at  the  current  cutoff.  Lowering  the 
GM  cutoff  by  five  points  (GM  =  92;  or  GM  =  83  &  GT  =  85)  would  increase  eligibility 
by  13  percentage  points,  while  the  average  male  Soldier  who  would  qualify  for  training 
would  have  essentially  the  same  chance  of  passing  (97%).  Raising  the  GM  cutoff  scores 
by  five  points  (GM  =  102;  or  GM  =  93  &  GT  =  85),  the  average  male  Soldier  who  would 
still  qualify  for  the  MOS  would  have  about  the  same  chance  of  passing  (99%),  but  fewer 
Soldiers  would  be  eligible  (71%). 37 


Table  2.  52D:  Probability  that  a  Soldier  (from  the  larger  Army  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model _ _ _ 


Cutoff: 

Cutoff: 

Cutoff: 

Cutoff: 

Cutoff: 

GM  =  87 

GM  =  92 

GM  =  97 

GM=  102 

GM  =  107 

-or- 

-or- 

-or- 

-or- 

-or- 

GM  =  78 

GM  =  83 

GM  =  88 

GM  =  93 

GM  =  98 

& 

& 

& 

& 

& 

in 

00 

II 

H 

O 

in 

00 

II 

H 

O 

in 

00 

II 

H 

O 

in 

00 

II 

H 

O 

O 

H 

II 

oo 

Percent  Eligible 
(Regular  Army) 

97.1% 

96.2% 

83.4% 

70. 7% 

57.6%> 

Mean:  GM 

104.65 

105.46 

106.88 

108.98 

111.30 

Mean:  GT 

106.68 

107.25 

108.13 

109.40 

110.79 

Passing  rates: 

High  School  Male 

97.3% 

97.5% 

97.7% 

97.9% 

98.2% 

77  Appendix  C  shows  the  percent  of  Soldiers  who  are  eligible  according  to  primary  and  alternative  cutoff 
criteria. 
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Figure  1  shows  the  relationship  between  GM  and  the  probability  of  passing  for 
high  school  educated  male  Soldiers,  including  upper  and  lower  bounds  based  upon  the 
standard  error  of  the  estimated  GM  coefficient.  For  a  particular  GM  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
corresponding  probability  of  passing.  For  example,  a  GM  value  of  100  corresponds  to  a 
passing  probability  of  about  97%  for  a  high  school  educated  male  Soldier. 


Figure  1:  Predicted  Probability  of  Passing:  Regular  Army  High  School  Male 


School  /  Male  —  —  HSM  Upper  —  ~  HSM  Lower 
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Appendix  A:  Soldier  Characteristics  (52D) 


Gender 

Outcome 

Male 

Female 

fail 

2.4% 

3.2% 

pass 

97.6% 

96.8% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

1.3% 

2.7% 

1.9% 

pass 

98.7% 

97.3% 

97.5% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

3.0% 

1 .3% 

2.5% 

pass 

97.0% 

98.7% 

97.5% 
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Appendix  B 


Table  la.  52D:  Results  of 
prediction  model:  GM  anc 

the  binary  logistic 

1  GT  model 

Chi-  Square 

38.048*** 

Log  Likelihood 

735.919 

Nagelkerke  R  Square 

.055 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-2.219 

.109 

2.841 

GED 

.478 

1.613 

.595 

COLLEGE 

.027 

1.027 

.002 

GUARD 

.106 

1.111 

.125 

RESERVE 

.626 

1.870 

2.874 

GENDER 

.184 

1.202 

.345 

GM 

.063 

1.065 

15.658*** 

GT 

-.007 

.993 

.118 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 
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Appendix  C:  Percent  Eligible 


Table  2a. 

Percent  eligible  under  primary  and  alternative  cutoff 
criteria. 

Cutoff 

Qualify 

Both 

Qualify 

GM 

Only 

Qualify 

GM/GT 

Only 

Qualify 

Total 

Do  Not 
Qualify 

-10 

86.3% 

5.0% 

5.8% 

97.1% 

2.9%o 

-5 

77.0% 

3.3% 

12.0% 

96.2% 

7.8% 

Current 

65.1% 

1.8% 

16.4% 

83.4% 

16.6% 

+5 

50.2% 

0.8% 

19.7% 

70. 7% 

29.3% 

+  10 

37.0% 

0.3% 

20.2% 

57.6% 

42.4% 
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63H:  Track  Vehicle  Repairer 


The  final  sample  included  1205  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (96.6%)  or  failed  for  academic  reasons  (3.4%)  were 
included  in  the  analysis  sample  of  1205  Soldiers  because  we  assume  that  non-completion 
for  non-academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  of  the  Soldiers  were  male  (91%,  9%  female),  most  had  a  high 
school  diploma  but  not  more  education  (92%,  4%  some  college,  3%  GED  or  less),  and 
the  greatest  number  were  from  Regular  Army  (82%,  17%  National  Guard,  1%  Army 
Reserve).  The  governing  AA  composites,  Mechanical  Maintenance  (MM)  and  General 
Technical  (GT),  for  this  MOS  have  a  cutoff  scores  of  92  and  85  respectively;  the  sample 
mean  for  MM  is  106.78  (standard  deviation  =  12.200)  and  for  GT  isl04.69  (standard 
deviation  =  10.784).  The  MM  and  GT  composites  are  correlated  at  .609.  See  the 
Appendix  for  a  description  of  Soldier  characteristics  for  this  MOS  training  sample. 


Table  1.  63H:  Results  of  the  full  binary  logistic 
prediction  model _ _ 


Chi-  Square 

19  794*** 

Log  Likelihood 

338.010 

Nagelkerke  R  Square 

.063 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-3.042 

.048 

3.078 

GUARD 

1.339 

4.050 

3.650 

GENDER 

.374 

1.453 

.355 

MM 

.032 

1.033 

3.551 

GT 

.028 

1.029 

2.945 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 
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A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  model,  including  the  GT  and  MM  composites,  Army 
component  (GUARD  =  National  Guard)  and  gender  (1  =  female),  accounts  for  about  six 
percent  of  the  variation  in  the  dependent  variable.  This  model  is  statistically  significant 
(X  =  19.794,  P  =  .001),  with  moderate  explanatory  power  (Nagelkerke  R~  =  .063). 

While  there  are  no  statistically  significant  effects  for  any  of  the  variables,  both  GT 
and  MM  approach  significance.  The  model  suggests  that  having  a  higher  GT  composite 
score  increases  the  odds  of  passing.  At  the  mean  GT  score,  an  increase  of  one  point  is 
associated  with  an  increase  of  about  three  percent  in  the  odds  of  passing  the  course,  and  a 
five-point  increase  in  GT  would  increase  the  odds  of  passing  by  15%.  The  model  also 
suggests  that  having  a  higher  MM  composite  score  increases  the  odds  of  passing.  At  the 
mean  MM  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  three 
percent  in  the  odds  of  passing  the  course,  and  a  five-point  increase  in  GT  would  increase 
the  odds  of  passing  by  19%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  on  the  first  attempt  based  upon 
the  binary  logistic  model.  These  were  calculated  using  the  formula  for  finding  probability 
in  a  binary  logistic  model,  which  is 

qw'x)  /  (1  +  e(P’x>). 

The  following  analysis  is  confined  to  male  Soldiers  from  the  Regular  Army  (though 
results  would  be  similar  for  other  demographic  groups).  Based  on  the  model,  when  the 
cutoff  score  is  at  its  current  level  (GT  =  85,  MM  =  92),  Regular  Army  Soldiers  with 
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average  GT  and  MM  scores  (GT  =  108.38,  MM  =  108.64)  have  approximately  a  98% 
chance  of  passing.  Currently,  about  82%  of  Soldiers  are  eligible  for  MOS  63H 
assignment  at  the  current  cutoff  (GT  =  85,  MM  =  92).  Lowering  the  MM  cutoff  by  five 
points  (GT  =  85,  MM  =  87)  would  increase  eligibility  by  8  percentage  points  (to  89%) 
and  the  average  Soldier  who  would  qualify  for  training  would  have  essentially  the  same 
chance  of  passing  (97%).  Raising  the  cutoff  scores  by  five  points  (GT  =  90,  MM  =  97), 
the  average  Soldier  would  have  essentially  the  same  chance  of  passing  (98%),  but 
eligibility  would  fall  by  about  15  percentage  points. 


Table  2.  63H:  Probata 
population)  will  pass  t 

lity  that  a  Male  Soldier  (from  the  larger  Anny  contract 
le  course  based  on  the  binary  logistic  model 

Cutoff 
GT  =  85 
MM  =  82 

Cutoff 
GT  =  85 
MM  =  87 

Cutoff 
GT  =  85 
MM  =  92 

Cutoff 
GT  =  90 
MM  =  97 

Cutoff 

GT  =  95 
MM  =  102 

Percent  Eligible 
(Regular  Army) 

90.4% 

88.6% 

81.7% 

66. 7%> 

55.6% 

Mean:  GT 

106.79 

107.31 

108.38 

110.26 

111.51 

Mean:  MM 

105.29 

106.47 

108.64 

111.31 

113.94 

Passing  rate: 

Male 

97.3% 

97.1% 

97.6%o 

97.9%o 

98.1% 

Figure  1  shows  the  relationship  between  MM  and  the  probability  of  passing  for 
Male  Regular  Anny  Soldiers,  including  upper  and  lower  bounds  based  upon  the  standard 
error  of  the  estimated  MM  coefficient  .  For  a  particular  MM  score,  trace  a  vertical  line 
up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  detennine  the  corresponding 
probability  of  passing.  For  example,  a  MM  score  of  100  corresponds  to  a  passing 
probability  of  about  95%  for  a  high  school  educated,  Regular  Army  male  Soldier.  One 
can  see  that  Soldiers  with  a  relatively  low  MM  score  still  have  a  very  good  chance  of 


38  Based  upon  parameter  estimates  of  Appendix  Table  la. 
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Ch. 


passing  the  course.  As  MM  scores  increase,  the  chance  that  a  Soldier  will  pass  the  course 
increases  slightly. 


Figure  1:  Predicted  Probalility  of  Passing:  Male  Soldiers 


Appendix:  Additional  Regression  Models 


Table  la.  63H:  Results  of  the  full  binary  logistic 
prediction  model  excluding  GT _ 


Chi-  Square 

17.315*** 

Log  Likelihood 

17.315 

Nagelkerke  R  Square 

.056 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-1.764 

.171 

1.334 

GUARD 

1.369 

3.931 

3.501 

GENDER 

.538 

1.712 

.757 

MM 

.047 

1.049 

10.061** 

*  =  p  <  .05  ***  =  p<.001 

**  =  p  <  .01 


Table  lb.  63H:  Results  of  the  full  binary  logistic 
prediction  model  excluding  MM _ 


Chi-  Square 

16.018*** 

Log  Likelihood 

341.785 

Nagelkerke  R  Square 

.051 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-1.143 

.319 

.696 

GUARD 

1.438 

4.214 

3.870* 

GENDER 

.111 

1.117 

.033 

GT 

.042 

1.043 

9.706** 

*  =  p  <  .05  ***  =  p<.001 

**  =  p  <  .01 
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Appendix:  Soldier  Characteristics  (63H) 


Gender 

Male 

Female 

fail 

3.5% 

2.7% 

pass 

96.5% 

97.3% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

.0% 

3.6% 

2.0% 

pass 

100.0% 

96.4% 

98.0% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

.0% 

1 .0% 

pass 

100.0% 

99.0% 

225 


63  J:  Quartermaster  and  Chemical  Equipment  Repairer 


The  final  sample  included  1643  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (93.5%)  or  failed  for  academic  reasons  (6.5%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (75%,  25%  female),  most  had  a  high  school 
diploma  but  not  more  education  (87%,  4%  some  college,  9%  GED  or  less),  and  most 
were  from  the  Regular  Army  (45%,  29%  Guard,  26%  Reserve).  The  governing  AA 
composite,  Mechanical  Maintenance  (MM),  for  this  MOS  has  a  cutoff  score  of  92;  the 
sample  mean  is  98.89  (standard  deviation  =  10.940).  This  MOS  has  an  alternative 
governing  AA  composite  cutoff,  where  soldiers  with  an  MM  score  of  87  can  also  qualify 
if  they  have  a  General  Technical  (GT)  score  of  85;  the  sample  mean  for  GT  is  100.76 
(standard  deviation  =  10.746)  See  the  Appendix  for  a  description  of  Soldier 
characteristics  for  this  MOS  training  sample.  GT  and  MM  are  correlated  at  r  =  .658. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes.  The  model,  including  the  MM  and  GT  composites,  education  level  (GED  = 
GED  or  less  education,  COLLGE  =  some  college),  Army  component  (GUARD  = 
National  Guard,  RESERVE  =  Army  Reserve),  and  gender  (GENDER  =  female), 
accounts  for  about  10%  of  the  variation  in  the  dependent  variable.  This  model  is 
statistically  significant  (x  =  60.898,  P  <  .001),  and  is  a  significant  improvement  upon  the 
MM-only  model  reported  in  Appendix  B  (x  =  9. 186,  P  =  .002).  The  model  has  moderate 
explanatory  power  (Nagelkerke  R“  =  .096). 
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Table  2.  63J:  Results  of  the  binary  logistic 
prediction  model:  MM  and  GT  model 


Chi-  Square 

60.898** 

Log  Likelihood 

725.170 

Nagelkerke  R  Square 

.096 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-6.764 

.001 

21.325*** 

GED 

.033 

1.033 

.006 

COLLEGE 

-0.71 

.932 

.020 

GUARD 

.751 

2.118 

7.864** 

RESERVE 

.450 

1.568 

2.739 

GENDER 

.256 

1.292 

1.135 

MM 

.052 

1.054 

10.188*** 

GT 

.042 

1.043 

8.802** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 


There  are  statistically  significant  effects  for  the  MM  composite,  the  GT 
composite,  and  National  Guard  Membership.  The  model  suggests  that  having  a  higher 
MM  (GT)  composite  score  increases  the  odds  of  passing.  At  the  mean  MM  (GT)  score, 
an  increase  of  one  (one)  point  is  associated  with  an  increase  of  about  five  (four)  percent 
in  the  odds  of  passing  the  course,  and  a  five-point  increase  in  MM  (GT)  would  increase 
the  odds  of  passing  by  29%  (23%).  There  is  also  a  notable  effect  for  National  Guard 
members  where  Guardsmen  have  1 12%  higher  odds  of  passing. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model  that  includes  both  MM  and  GT.  These  were  calculated  using  the  formula  for 
finding  probability  in  a  binary  logistic  model,  which  is 

Q(fS’x)  /  (1  + 

Because  GED,  COLLEGE,  and  GENDER  were  not  statistically  significant,  this  analysis 
is  confined  to  the  modal  demographic — high  school  educated  male  Soldiers  (though 


227 


results  would  be  similar  for  any  demographic  combination).  Based  on  the  model,  when 
the  cutoff  score  is  at  its  current  levels  (MM  =  92,  or  GT  =  85  &  MM  =  87),  male  Soldiers 


with  a  high  school  diploma  and  average  composite  scores  (GT  =  106.36,  MM  =  104.63) 
have  approximately  a  96%  chance  of  passing.  Currently,  about  87%  of  Soldiers  are 
eligible  for  MOS  63  J  assignment  at  the  current  cutoff.  Lowering  the  MM  cutoff  by  five 
points  (MM  =  87,  or  GT  =  85  &  MM  =  82)  would  increase  eligibility  by  13  percentage 
points,  while  the  average  male  Soldier  who  would  qualify  for  training  would  have 
essentially  the  same  chance  of  passing.  Raising  the  cutoff  scores  by  five  points  (MM  = 
97,  or  GT  =  90  &  MM  =  92),  the  average  male  Soldier  who  would  still  qualify  for  the 
MOS  would  have  about  the  same  chance  of  passing,  but  fewer  Soldiers  would  be  eligible 
(66%)39. 


Table  2.  63  J:  Probability  that  a  Soldier  (from  the  larger  Anny  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model 

Cutoff: 
MM  =  82 

-or- 

MM  =  77 
& 

GT  =  85 

Cutoff: 
MM  =  87 

-or- 

MM  =  82 
& 

GT  =  85 

Cutoff: 
MM  =  92 

-or- 

MM  =  87 
& 

GT  =  85 

Cutoff: 
MM  =  97 

-or- 

MM  =  92 
& 

GT  =  85 

Cutoff: 
MM  =  102 

-or- 

MM  =  97 
& 

GT  =  85 

Percent  Eligible 
(Regular  Army) 

98.2% 

94.5% 

86.9% 

73.0% 

58.4% 

Mean:  MM 

104.63 

105.22 

106.43 

108.83 

111.51 

Mean:  GT 

106.36 

106.61 

107.19 

108.59 

110.25 

Passing  rates: 

High  school  male 

95.9% 

96.0% 

96.3% 

96.9% 

97.5% 

Figure  1  shows  the  relationship  between  MM  and  the  probability  of  passing  for 
high  school  educated  male  Soldiers,  including  upper  and  lower  bounds  based  upon  the 


39  Table  2  in  Appendix  C  shows  the  percent  of  Soldiers  who  are  eligible  according  to  primary  and 
alternative  cutoff  criteria. 
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standard  error  of  the  estimated  MM  coefficient  (for  the  model  in  Appendix  B).  For  a 
particular  MM  score,  trace  a  vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over 
to  the  axis  to  detennine  the  corresponding  probability  of  passing.  For  example,  a  MM 
value  of  100  corresponds  to  a  passing  probability  of  about  93%  for  a  high  school 
educated  male  Soldier. 


Figure  1:  Predicted  Probalility  of  Passing:  Male  High  School  Soldiers 


Regular  Army  -  -  -  Army  Upper  -  -  -  Army  Lower 
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Appendix  A:  Soldier  Characteristics  (63  J) 


Gender 

Outcome 

Male 

Female 

fail 

6.3% 

7.0% 

pass 

93.7% 

93.0% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

4.9% 

6.6% 

6.8% 

pass 

95.1% 

93.4% 

93.2% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

8.7% 

4.8% 

4.5% 

pass 

91.3% 

95.2% 

95.5% 
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Appendix  B:  Regression  Model  with  MM  Composite 


Table  la.  63J:  Results  of  the  binary  logistic  prediction 
model:  MM-only  model  _ 


Chi-  Square 

51.712*** 

Log  Likelihood 

734.355 

Nagelkerke  R  Square 

.081 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-5.197 

.006 

15.404*** 

GED 

-.003 

.997 

.000 

COLLEGE 

-.038 

.963 

.006 

GUARD 

.756 

2.129 

7.948** 

RESERVE 

.463 

1.588 

2.911 

GENDER 

.348 

1.416 

2.153 

MM 

.078 

1.081 

31.520 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 


The  model,  including  the  MM  composite,  education  level  (GED  =  GED  or  less 
education,  COLLGE  =  some  college),  Anny  component  (GUARD  =  National  Guard, 
RESERVE  =  Army  Reserve),  and  gender  (GENDER  =  female),  accounts  for  about  8%  of 
the  variation  in  the  dependent  variable.  This  model  is  statistically  significant  (%  =  51.712, 
P  <  .001),  and  has  moderate  explanatory  power  (Nagelkerke  R  =  .081). 

There  are  statistically  significant  effects  for  the  MM  composite  and  National 
Guard  Membership.  The  model  suggests  that  having  a  higher  MM  composite  score 
increases  the  odds  of  passing.  At  the  mean  MM  score,  an  increase  of  one  point  is 
associated  with  an  increase  of  about  eight  percent  in  the  odds  of  passing  the  course,  and  a 
five-point  increase  in  MM  would  increase  the  odds  of  passing  by  48%.  There  is  also  a 
notable  effect  for  National  Guard  members  where  Guardsmen  have  1 13%  higher  odds  of 
passing. 
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Appendix  C:  Percent  Eligible 


Table  2a. 

Percent  eligible:  breakdown  by  primary  and  alternative 
criteria 

Cutoff 

Qualify 

Both 

Qualify 

MM 

Only 

Qualify 

MM/GT 

Only 

Qualify 

Total 

Do  Not 
Qualify 

-10 

95.2% 

0.9% 

2.1% 

98.2% 

1.8% 

-5 

88.9% 

0.7% 

4.9% 

94.5% 

5.5% 

Current 

76.8% 

0.5% 

9.7% 

86.9% 

13.1% 

+5 

61.3% 

2.0% 

9.7% 

73.0% 

27.0% 

+  10 

46.2% 

3.0% 

9.2% 

58.4% 

41.6% 
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63 W:  Wheeled  Vehicle  Repairer 


The  final  sample  included  4,486  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (96.8%)  or  failed  for  academic  reasons  (3.2%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  this  sample,  most  were  male  (88%,  12%  female),  most  had  a  high  school 
diploma  but  not  more  education  (88%,  5%  some  college,  7%  GED  or  less),  and  most 
were  from  Regular  Army  (52%,  30%  National  Guard,  18%  Reserve).  The  governing  AA 
composites,  Mechanical  Maintenance  (MM)  and  General  Technical  (GT),  for  this  MOS 
have  cutoff  scores  of  92  and  85  respectively;  the  sample  mean  is  105.35  (standard 
deviation  =  12.195)  for  MM,  and  103.09  (standard  deviation  =  1 1.278)  for  GT.  The 
governing  AA  composites  are  correlated  at  .954  (p  <  .001).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 


Table  1.  63W:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

77.608*** 

Log  Likelihood 

1189.304 

Nagelkerke  R  Square 

.070 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-3.625 

.027 

12.871* 

GED 

.366 

1.441 

.591 

COLLEGE 

-.055 

.947 

.014 

GUARD 

.435 

1.545 

4.055* 

RESERVE 

1.076 

2.933 

10.089*** 

GENDER 

-.061 

.941 

.059 

MM 

.032 

1.032 

8.959** 

GT 

.036 

1.036 

10  447*** 

*  =  D  <  .05  **  =  n  <  .01  ***  =  D<-001 
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outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  a  model,  including  the  MM  and  GT  composites,  education  level 
(GED  =  GED  or  less  education,  COLLEGE  =  some  college  education),  Army  component 
(GUARD  =  National  Guard;  RESERVE  =  Army  Reserve)  and  gender  (GENDER  = 
female),  accounts  for  about  six  percent  of  the  variation  in  the  dependent  variable.  This 
model  is  statistically  significant  (%2  =  77.608,  P  <  .001),  and  has  somewhat  limited 
explanatory  power  (Nagelkerke  R"  =  .070). 

There  are  statistically  significant  effects  for  the  MM  composite,  the  GT 
composite,  National  Guard  membership,  and  Reserve  membership.  The  model  suggests 
that  having  a  higher  MM  composite  score  increases  the  odds  of  passing.  At  the  mean 
MM  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  three  percent 
in  the  odds  of  passing  the  course,  and  a  five-point  increase  in  MM  would  increase  the 
odds  of  passing  by  17%.  The  model  also  suggests  that  having  a  higher  GT  composite 
score  increases  the  odds  of  passing.  At  the  mean  GT  score,  an  increase  of  one  point  is 
associated  with  an  increase  of  about  four  percent  in  the  odds  of  passing  the  course,  and  a 
five-point  increase  in  GT  would  increase  the  odds  of  passing  by  20%.  There  is  also  a 
noteworthy  effect  for  GUARD  (RESERVE):  the  odds  of  passing  the  course  for 
Guardsmen  (Reservists)  are  55%  (193%)  higher  than  that  of  the  average  Regular  Anny 
Soldier. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 
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Q(P’x)  /  (1  +  Q(P’x)y 

This  analysis  is  confined  to  the  modal  demographics:  male  Soldiers  with  a  high  school 
diploma  from  the  Regular  Army.  Based  on  the  model,  when  the  cutoff  score  is  at  its 
current  level  (MM  =  92,  GT  =  85),  male  soldiers  with  average  scores  (MM  =  108.64,  GT 
=  108.38)  have  approximately  a  98%  chance  of  passing.  Currently,  about  82%  of  Soldiers 
are  eligible  for  MOS  63W  assignment  at  the  current  cutoff  (MM  =  92,  GT  =  85).  Raising 
the  cutoff  by  five  points  (MM  =  97,  GT  =  90),  the  average  male  Soldier  who  would 
qualify  for  training  would  have  essentially  the  same  chance  of  passing,  while  eligibility 
would  decrease  by  15  percentage  points  (to  67%).  Lowering  the  cutoff  score  by  five 
points  (MM  =  87,  GT  =  80)  would  increase  eligibility  by  approximately  12  percentage 
points,  and  the  average  male  Soldier  would  have  essentially  the  same  chance  of  passing 
as  he  would  at  the  current  cutoff  score. 


Table  3.  63 W:  Probab 
population)  will  pass  t 

ility  that  a  male  Soldier  (from  the  larger  Anny  contract 
le  course  based  on  the  binary  logistic  model 

Cutoff  = 
82/75 

Cutoff = 
87/80 

Cutoff = 
92/85 

Cutoff = 
97/90 

Cutoff = 
102/95 

Percent  Eligible 
(Regular  Army) 

98.8% 

94.2% 

57.7% 

66. 7% 

49.5% 

Mean:  MM 

105.19 

106.39 

108.64 

111.31 

114.19 

Mean:  GT 

106.58 

106.58 

108.38 

110.26 

112.34 

Passing  rates: 

High  School  Male 

97.3% 

97.4% 

97.7% 

98.0% 

98.3% 

Figure  1  shows  the  relationship  between  MM  and  the  probability  of  passing  for 
male  Regular  Army  Soldiers,  based  upon  a  regression  model  that  excludes  GT.40  Figure 
1  also  includes  upper  and  lower  bounds  based  upon  the  standard  error  of  the  estimated 


40  As  shown  in  the  Appendix.  Models  that  include  either  MM  or  GT  have  nearly  as  good  a  fit  as  one 
containing  both,  and  lend  themselves  more  easily  to  graphing  and  interpretation. 
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MM  coefficient.  For  a  particular  MM  score,  trace  a  vertical  line  up  to  the  curve,  and  then 
a  horizontal  line  over  to  the  axis  to  determine  the  corresponding  probability  of  passing. 
For  example,  an  MM  value  of  100  corresponds  to  a  passing  probability  of  about  95%  for 
a  high  school  educated,  Regular  Anny  male  Soldier.  One  can  see  that  male  Soldiers  with 
a  relatively  low  MM  score  stand  a  reasonably  good  chance  of  passing  the  course.  As  the 
MM  (or  GT)  composite  scores  increase,  the  chance  that  a  Soldier  will  pass  the  course 
increases,  essentially  reaching  100%  just  above  the  current  cutoff. 


Figure  1 :  Predicted  Probability  of  Passing:  Regular  Army  High  School  Male  Soldiers 


High  School  /  Male  —  —  HSM  Upper  —  ~  HSM  Lower 
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Appendix:  Additional  Regression  Models 


Table  1A.  63W:  Results  of  the  binary  logistic 
prediction  model  excluding  GT 

Chi-  Square 

67.306*** 

Log  Likelihood 

119.605 

Nagelkerke  R  Square 

.061 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-2.176 

.114 

6.495** 

GED 

.369 

1.446 

.603 

COLLEGE 

.088 

1.092 

.035 

GUARD 

.440 

1.553 

4.679* 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 


Table  IB.  63W:  Results  of  the  binary  logistic 
prediction  model  excluding  MM 

Chi-  Square 

68.135*** 

Log  Likelihood 

1198.777 

Nagelkerke  R  Square 

.061 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-2.205 

.110 

6.991** 

GED 

.381 

1.464 

.643 

COLLEGE 

-.015 

.985 

.001 

GUARD 

.418 

1.519 

4.223* 

RESERVE 

1.094 

2.987 

10.266*** 

GENDER 

-.301 

.740 

1.584 

GT 

.054 

1.055 

40.427*** 

*  =  p<  .05 

**  =  p  <  .01 

*** 

=  p  <  .00] 
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Appendix:  Soldier  Characteristics  (63W) 


Gender 

Outcome 

Male 

Female  | 

fail 

3.1% 

4.0% 

pass 

96.9% 

96.0% 

Education  Level  I 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

1.6% 

3.4% 

2.3% 

pass 

98.4% 

96.6% 

97.7% 

Component  I 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

4.1% 

1 .3% 

2.7% 

pass 

95.9% 

98.7% 

97.3% 
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91A:  Medical  Equipment  Repairer 


The  final  sample  included  241  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (44%)  or  failed  for  academic  reasons  (56%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  of  the  solders  were  male  (84%,  16%  female),  most  had  a  high 
school  diploma  but  not  more  education  (80%,  18%  some  college,  2%  GED  or  less),  and 
most  were  from  Army  Reserve  (50%,  41%  Regular  Army,  10%  National  Guard).  The 
governing  AA  composite,  Electronics  Repair  (EL),  for  this  MOS  has  a  cutoff  score  of 
107;  the  sample  mean  is  1 16.74  (standard  deviation  =  10.380).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 


Table  1.  91  A:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

1 1.351* 

Log  Likelihood 

319.248 

Nagelkerke  R  Square 

.062 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-4.231 

.015 

3.774 

COLLEGE 

.083 

1.086 

.053 

RESERVE 

.413 

1.512 

2.063 

GUARD 

-.293 

.746 

.359 

GENDER 

-.683 

.505 

3.163 

EL 

.033 

1.034 

3.194x 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .005  x  p  =  .074 
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Table  1  and  indicate  that  the  model,  including  the  EL  composite,  education  level 
(COLLEGE),  Army  component  (GUARD  =  National  Guard,  RESERVE  =  Anny 
Reserve)  and  gender  (GENDER  =  female),  accounts  for  about  six  percent  of  the  variation 
in  the  dependent  variable.  This  model  is  statistically  significant  (x  =  1 1.351,  P  =  .045), 
but  it  has  somewhat  limited  explanatory  power  (Nagelkerke  R"  =  .062). 

The  EL  composite  is  not  statistically  significant  at  the  usual  p=.05  level,  but  is 
significant  at  the  p  =  .074  level.  The  model  suggests  that  having  a  higher  EL  composite 
score  increases  the  odds  of  passing.  At  the  mean  EL  score,  an  increase  of  one  point  is 
associated  with  an  increase  of  about  4%  in  the  odds  of  passing  the  course,  and  a  five- 
point  increase  in  EL  would  increase  the  odds  of  passing  by  25%.  There  is  also  a  near 
significant  effect  for  GENDER  where  women  have  49%  lower  odds  of  passing  when 
compared  to  men. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

q(P’x)  /  (1  +  q<p’x}). 

Based  on  the  model,  when  the  cutoff  score  is  at  its  current  level  (EL  =  107),  male 
Soldiers  with  an  average  EL  score  (EL  =  1 17.05)  have  approximately  a  41%  chance  of 
passing  on  the  first  attempt,  while  female  Soldiers  have  a  26%  chance  of  passing. 
Currently,  about  47%  of  Soldiers  are  eligible  for  MOS  91 A  assignment  at  the  current 
cutoff  (EL  =  107).  Lowering  the  cutoff  by  five  points  (EL  =  102)  increases  eligibility  by 
15  percentage  points  (to  62%),  while  the  average  Soldier  who  would  qualify  for  training 
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would  have  practically  the  same  chance  of  passing.  Raising  the  cutoff  score  by  five 
points  (EL  =  112),  the  average  Soldier  would  have  a  somewhat  higher  chance  of  passing 
but  eligibility  would  be  reduced  by  14  percentage  points  (to  33%). 


Table  2.  91  A:  Probability  that  a  Soldier  (from  the  larger  Army  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model _ _ _ 


Cutoff = 

Cutoff = 

Cutoff = 

Cutoff = 

Cutoff = 

97 

102 

107 

112 

117 

Percent  Eligible 
(Regular  Army) 

76.09% 

62.2% 

47.4% 

32.8% 

22.9%> 

Mean 

111.34 

114.03 

117.05 

120.42 

123.88 

Passing  rates: 

Male  High  School 

36.4% 

38.5% 

40.9% 

43.6% 

46.4% 

Female  High  School 

22.4% 

24.0% 

25.9% 

28.1% 

30.5% 

Figure  1  shows  the  relationship  between  EL  and  the  probability  of  passing  for 
male  and  female  Soldiers,  including  upper  and  lower  bounds  based  upon  the  standard 
error  of  the  estimated  EL  coefficient.  For  a  particular  EL  score,  trace  a  vertical  line  up  to 
the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the  corresponding 
probability  of  passing.  For  example,  an  EL  value  of  100  corresponds  to  a  passing 
probability  of  about  28%  for  a  high  school  educated,  Regular  Army  male  Soldier.  One 
can  see  that  Soldiers  with  scores  in  the  vicinity  of  the  cutoff  level  have  a  low  chance  of 
passing  the  course.  Apparently,  this  is  a  relatively  difficult  course  and  the  estimated 
model,  based  on  limited  information,  does  not  provide  much  insight  into  the  responsible 
factors. 
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Figure  1:  Predicted  Probalility  of  Passing:  Regular  Army  Soldiers 


Appendix:  Soldier  Characteristics  (91  A) 


Gender 

Male 

Female 

fail 

pass 

53.5% 

46.5% 

69.2% 

30.8% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

pass 

56.5% 

43.5% 

54.5% 

45.5% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

pass 

60.2% 

39.8% 

50.8% 

49.2% 

65.2% 

34.8% 
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91B:  Medical  Specialist41 

The  final  sample  included  5054  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (92.5%)  or  failed  for  academic  reasons  (7.5%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (58%,  42%  female),  most  had  a  high  school 
diploma  but  not  more  education  (85%,  1 1%  some  college,  4%  GED  or  less),  and  most 
were  from  were  from  the  Regular  Anny  (57%,  22%  National  Guard,  26%  Reserve).  The 
governing  AA  composite,  Skilled  Technical  (ST),  for  this  MOS  has  a  cutoff  score  of  95; 
the  sample  mean  is  108.34  (standard  deviation  =  1 1.821).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt. 

The  estimation  results  are  reported  in  Table  1  and  indicate  that  a  model,  including 
the  ST  composite,  education  level  (GED  =  GED  or  less  than  high  school  diploma; 
COLLEGE  =  some  college), Army  component  (GUARD  =  National  Guard;  RESERVE  = 
Army  Reserve)  and  gender  (GENDER  =  female),  accounts  for  about  13%  of  the  variation 
in  the  dependent  variable.  This  model  is  statistically  significant  (%2  =  269.735,  P  <  .001), 
and  has  moderate  explanatory  power  (Nagelkerke  R"  =  .125). 


41  As  of  FY’02,  MOS  91B  was  closed  and  merged  with  MOS  91W.  Thus,  this  analysis  applies  only  to 
Soldiers  trained  before  the  merger. 
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Table  1.  9  IB:  Results  of  the  binary  logistic 
prediction  model 

Chi-  Square 

269.735*** 

Log  Likelihood 

2432.666 

Nagelkerke  R  Square 

.125 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-4.706 

.009 

54.022*** 

GED 

-.030 

.970 

.008 

COLLEGE 

1.162 

3.198 

14.457*** 

RESERVE 

.706 

2.026 

16.687*** 

GUARD 

.847 

2.332 

24.812*** 

GENDER 

.620 

1.859 

27.820*** 

ST 

.063 

1.065 

105.984*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p<  .01 

[ 

There  is  a  statistically  significant  effect  for  the  ST  composite.  The  model 
suggests  that  having  a  higher  ST  composite  score  increases  the  odds  of  passing.  At  the 
mean  ST  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  6%  in  the 
odds  of  passing  the  course,  and  a  five-point  increase  in  ST  would  increase  the  odds  of 
passing  by  32%.  There  are  also  statistically  significant  effects  for  COLLEGE  (220% 
increased  odds  of  passing),  Reserve  membership  (100%  increased  odds  of  passing), 
National  Guard  membership  (133%  increased  odds  of  passing),  and  GENDER  (86% 
increased  odds  of  passing  for  women). 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

Q(P’x)  /  (1  +  Q(P’x)y 
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Because  GED  was  not  statistically  significant,  this  analysis  is  confined  to  male  and 
female  Soldiers  with  a  high  school  diploma  or  some  college  education  from  the  Regular 
Army.  Based  on  the  model,  when  the  cutoff  score  is  at  its  current  level  (ST  =  95),  male 
Regular  Army  Soldiers  with  a  high  school  diploma  (the  modal  categories)  and  an  average 
ST  score  (ST  =  1 10.70)  have  approximately  a  93%  chance  of  passing.  Currently,  about 
83%  of  Soldiers  are  eligible  for  MOS  9  IB  assignment  at  the  current  cutoff  (ST  =  95). 
Lowering  the  cutoff  by  five  points  (ST  =  90)  increases  eligibility  by  10  percentage  points 
(to  93%),  while  the  average  male,  high  school  educated  Soldier  who  would  qualify  for 
training  would  have  almost  the  same  chance  of  passing  (92%).  Raising  the  cutoff  score 


Table  2.  9 IB:  Probabi 
population)  will  pass  t 

lity  that  a  Soldier  (from  the  larger  successful  applicant 
le  course  based  on  the  binary  logistic  model 

Cutoff  = 

85 

Cutoff = 

90 

Cutoff = 

95 

Cutoff = 
100 

Cutoff = 
105 

Percent  Eligible 
(Regular  Army) 

98.9% 

92.8% 

82.8% 

70. 6% 

55.8% 

Mean 

107.18 

108.67 

110.70 

113.07 

115.93 

Passing  rates: 

High  School  Male 

91.5% 

92.1% 

92.9% 

93.7% 

94.6% 

College  Male 

97.4% 

97.6% 

97.8% 

98.1% 

98.4% 

High  School  Female 

96.2% 

96.5% 

96.8% 

97.2% 

97.6% 

College  Female 

98.9% 

98.9% 

99.1% 

99.2% 

99.3% 

by  five  points  (ST  =  100),  the  average  qualifying  Soldier  would  have  a  marginally  higher 
chance  of  passing  (94%)  but  eligibility  would  fall  by  about  1 1  percentage  points  (to 
71%). 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
male  Regular  Army  Soldiers  by  education,  including  upper  and  lower  bounds  based  upon 
the  standard  error  of  the  estimated  ST  coefficient.  For  a  particular  ST  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
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corresponding  probability  of  passing.  For  example,  an  ST  value  of  100  corresponds  to  a 
passing  probability  of  about  87%  for  a  high  school  educated.  Regular  Army  male  Soldier. 
The  graph  illustrates  the  strength  of  the  effect  for  some  college  over  the  lower  range  of 
aptitude  scores. 


Figure  1:  Predicted  Probalility  of  Passing:  Male  Soldiers 


—  High  School  Male  -  -  -  High  School  Male  Upper  -  -  -  High  School  Male  Lower 

College  Male  —  -  College  Male  Upper  —  -  College  Male  Upper 


Figure  2  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
female  Regular  Army  Soldiers  by  education,  including  upper  and  lower  bounds  based 
upon  the  standard  error  of  the  estimated  ST  coefficient.  The  pattern  is  much  the  same  as 
was  apparent  for  male  Soldiers. 

Figure  3  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
Regular  Army  Soldiers  by  education  and  gender. 
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Cha 


Figure  2:  Predicted  Probalmty  of  Passing:  Female  Soldiers 


70  75  80  85  90  95  100  105  110  115  120  125  130  135  140  145  150  155  160 

ST  Composite  Scores 


High  School  Female 
College  Female 


■  High  School  Female  Upper  - 
College  Female  Upper  — 


-  High  School  Female  Lower 

-  College  Female  Lower 


Figure  3:  Probability  of  Passing:  Education  factored  by  Sex 
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Appendix:  Soldier  Characteristics  (9 IB) 


Gender 

Male 

Female 

Outcome 

fail 

8.6% 

6.1% 

pass 

91.4% 

93.9% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

pass 

6.0% 

94.0% 

8.3% 

91 .7% 

2.6% 

97.4% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

pass 

10.0% 

90.0% 

4.1% 

95.9% 

4.4% 

95.6% 
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91C:  Health  Care — Practical  Nurse 


The  final  sample  included  475  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (83.2%)  or  failed  for  academic  reasons  (16.8%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

The  sample  was  approximately  split  by  gender  (53%  male,  47%  female),  most 
had  a  high  school  diploma  but  not  more  education  (81%,  18%  some  college,  1%  GED  or 
less),  and  most  were  from  Regular  Army  (59%,  1%  National  Guard,  40%  Reserve).  The 
governing  AA  composites,  Skilled  Technical  (ST)  and  General  Technical  (GT),  for  this 
MOS  have  cutoff  scores  of  102  and  110,  respectively;  the  sample  mean  is  117.19 
(standard  deviation  =  7.907)  for  ST,  and  1 17.57  (standard  deviation  =  5.815)  for  GT. 
There  is  a  significant,  strong  correlation  between  ST  and  GT  (r  =  .678,  P  <  .001).  See  the 
Appendix  for  a  description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 


Table  1.  9 1C:  Results  of  the  binary  logistic 
prediction  model 

Chi-  Square 

39.782*** 

Log  Likelihood 

390.923 

Nagelkerke  R  Square 

.135 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-11.521 

.000 

14.490*** 

COLLEGE 

.308 

1.360 

.672 

RESERVE 

-.041 

.960 

.024 

GENDER 

.976 

2.654 

12.511*** 

ST 

.095 

1.100 

15.590*** 

GT 

.015 

1.015 

.217 

*  =  p  <  .05  ***  =  p<.00] 

**  =  p  <  .005 

[ 
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outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  a  model,  including  the  ST  and  GT  composites,  education  level 
(COLLEGE  =  some  college),  Anny  component  (RESERVE  =  Army  Reserve),  and 
gender  (GENDER  =  female),  accounts  for  about  14%  of  the  variation  in  the  dependent 
variable.  This  model  is  statistically  significant  (x2  =  39.782,  P  <  .001),  and  has  moderate 
explanatory  power  (Nagelkerke  R2  =  .  135). 42 

There  are  statistically  significant  effects  for  the  ST  composite  and  GENDER.  The 
model  suggests  that  having  a  higher  ST  composite  score  increases  the  odds  of  passing. 

At  the  mean  ST  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about 
10%  in  the  odds  of  passing  the  course,  and  a  five-point  increase  in  ST  would  increase  the 
odds  of  passing  by  61%.43  There  is  also  a  noteworthy  effect  for  GENDER:  female 
Soldiers  have  increased  odds  of  passing  the  course  (165%  higher). 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

Q(f)’x)  /  (1  +  Q(P’x)y 


42  Given  the  strong  correlation  between  ST  and  GT,  we  also  report  regression  results  for  GT  and  ST  in 
separate  regressions  in  the  Appendix.  The  ST  model  has  somewhat  better  explanatory  power,  though  the 
GT  model  is  also  statistically  significant.  With  both  composites  in  the  same  equation,  the  ST  composite 
dominates. 

43  As  shown  in  the  Appendix,  the  same  effects  would  be  seen  with  the  GT  composite. 
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Because  GUARD  and  RESERVE  were  not  statistically  significant,  this  analysis  is 
confined  to  the  modal  demographics  factored  by  GENDER:  male  and  female  Soldiers 
from  the  Regular  Army,  but  the  results  would  apply  to  the  average  Guard  and  Reserve 
Soldier  as  well.  Based  on  the  model,  when  the  cutoff  score  is  at  its  current  level  (ST  = 
102,  GT  =  110),  male  soldiers  with  average  ST  and  GT  scores  (ST  =  1 17.40,  GT  = 

1 17.85)  have  approximately  an  82%  chance  of  passing,  while  their  female  counterparts 
have  approximately  a  93%  chance  of  passing.  Currently,  about  33%  of  Soldiers  are 
eligible  for  MOS  92C  assignment  at  the  current  cutoff  level.  Lowering  the  cutoff  score 
by  five  points  (ST  =  97,  GT  =  105)  would  increase  eligibility  by  15  percentage  points  (to 
47%),  while  average  male  and  female  Soldier  passing  rates  would  drop  by  3  -  5 


Table  2.  9 1C:  Probability  that  a  Soldier  (from  the  larger  Anny  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model 

Cutoff  = 
92/100 

Cutoff = 
97/105 

Cutoff = 
102/110 

Cutoff = 
107/110 

Cutoff = 
112/110 

Percent  Eligible 
(Regular  Army) 

65.1% 

47.2% 

52.7% 

28.2% 

22.7% 

Mean:  ST 

112.90 

115.91 

118.87 

120.03 

122.07 

Mean:  GT 

117.62 

117.65 

118.51 

118.88 

119.68 

Passing  rates: 

Male 

72.5% 

77.8% 

82.5% 

84.1% 

86.7% 

Female 

87.2% 

90.3% 

92.6% 

93.3% 

94.5% 

percentage  points.  Raising  the  ST  cutoff  by  five  points,  but  keeping  GT  constant  at  1 10 
(ST  =  107,  GT  =  110),  would  increase  average  male  and  female  passing  rates  slightly  but 
eligibility  would  decrease  by  five  percentage  points  (to  28%). 
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Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
male  and  female  Regular  Army  Soldiers  based  on  the  regression  in  Table  la  (Appendix). 


Figure  1 :  Predicted  Probability  of  Passing:  Male  vs.  Female  Soldiers 


|—  Male  —  -  Male  Upper  —  ■  Male  Lower  Female  -  -  -  Female  Upper  -  -  -  Female  Lower  | 

It  includes  upper  and  lower  bounds  based  upon  the  standard  error  of  the  estimated  ST 
coefficient.  For  a  particular  ST  score,  trace  a  vertical  line  up  to  the  curve,  and  then  a 
horizontal  line  over  to  the  axis  to  determine  the  corresponding  probability  of  passing. 

For  example,  an  ST  score  of  100  corresponds  to  a  passing  probability  of  about  72%  for  a 
high  school  educated,  Regular  Anny  female  Soldier. 
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Appendix:  Soldier  Characteristics  (9 1C) 


Gender 

Male 

Female 

Outcome 

fail 

20.6% 

12.6% 

pass 

79.4% 

87.4% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

pass 

.0% 

100.0% 

18.2% 

81 .8% 

1 1 .5% 

88.5% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

pass 

16.7% 

83.3% 

16.8% 

83.2% 

33.3% 

66.7% 
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Appendix:  Additional  Regression  Models 


Table  la.  91C:  Results  of  the  binary  logistic 
prediction  model  with  only  the  ST  composite. 


Chi-  Square 

39.546*** 

Log  Likelihood 

391.141 

Nagelkerke  R  Square 

.134 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-10.597 

.000 

21.948*** 

COLLEGE 

.317 

1.373 

.716 

RESERVE 

-.039 

.961 

.023 

GENDER 

.991 

2.693 

13.059*** 

ST 

.102 

1.107 

27.020*** 

*  =  p  <  .05  ***  =  p  <  00 

**  =  p  <  .005 


Table  lb.  9 1C:  Results  of  the  binary  logistic 
prediction  model  with  only  the  GT  composite. 


Chi-  Square 

23  197*** 

Log  Likelihood 

407.509 

Nagelkerke  R  Square 

.080 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-9.306 

.000 

10.870** 

COLLEGE 

.360 

1.434 

.948 

RESERVE 

-.026 

.974 

.010 

GENDER 

.680 

1.974 

6.794* 

GT 

.091 

1.095 

13.897*** 

*  =  p  <  .05  ***  =  p  <  00 

**  =  p  <  .005 
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91D:  Operating  Room  Specialist 


The  final  sample  included  1063  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (85%)  or  failed  for  academic  reasons  (15%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  female  (53%,  47%  male),  most  had  a  high  school 
diploma  but  not  more  education  (83%,  12%  some  college,  6%  GED  or  less),  and  most 
were  evenly  divided  between  the  Regular  Army  and  the  Reserves  (48%,  48%,  3% 
National  Guard).  The  governing  AA  composite,  Skilled  Technical  (ST),  for  this  MOS  has 
a  cutoff  score  of  92;  the  sample  mean  is  105.00  (standard  deviation  =  9.857).  See  the 
Appendix  for  a  description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 


Table  1.  9  ID:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

98.320*** 

Log  Likelihood 

795.314 

Nagelkerke  R  Square 

.155 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-9.009 

.000 

39.299*** 

GED 

.632 

1.882 

1.585 

COLLEGE 

1.255 

3.509 

8.227** 

RESERVE 

.169 

1.184 

.801 

GUARD 

.104 

1.110 

.039 

GENDER 

.903 

2.467 

23.073*** 

ST 

.098 

1.103 

49.873*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 
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Table  1  and  indicate  that  a  model,  including  the  ST  composite,  education  level  (GED  = 
GED  or  less  than  high  school  diploma;  COLLEGE  =  some  college),  Army  component 
and  gender  (GENDER  =  female),  accounts  for  about  16%  of  the  variation  in  the 
dependent  variable.  This  model  is  statistically  significant  (x  =  98.320,  P  <  .001),  and  has 
moderate  explanatory  power  (Nagelkerke  R2  =  .155). 

There  is  a  statistically  significant  effect  for  the  ST  composite.  The  model 
suggests  that  having  a  higher  ST  composite  score  increase  the  odds  of  passing.  At  the 
mean  ST  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  10%  in 
the  odds  of  passing  the  course,  and  a  five-point  increase  in  ST  would  increase  the  odds  of 
passing  by  63%.  There  are  also  statistically  significant  effects  for  COLLEGE  and 
GENDER,  with  increased  odds  of  passing  the  course  at  251%  and  147%  (for  females), 
respectively. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

Q(p’x)  /  (1  + 

The  following  analysis  is  focused  upon  male  and  female  Soldiers  with  a  high  school 
diploma  or  some  college  education  from  the  Regular  Army.  Based  on  the  model,  when 
the  cutoff  score  is  at  its  current  level  (ST  =  92),  male  Soldiers  with  a  high  school  diploma 
and  an  average  ST  score  (ST  =  109.45)  have  approximately  an  85%  chance  of  passing; 
their  female  counterparts  with  a  high  school  education  would  have  approximately  a  93% 
chance  of  passing.  Currently,  about  89%  of  Soldiers  are  eligible  for  MOS  9  ID 
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assignment  at  the  current  cutoff  (ST  =  92).  Lowering  the  cutoff  by  five  points  (ST  =  87) 
would  increase  eligibility  by  eight  percentage  points  (to  97%)  while  the  average  male 
Soldier  who  would  qualify  for  training  would  have  a  slightly  lower  chance  of  passing 
(82%);  the  chance  of  the  average  female  Soldier  passing  would  remain  unchanged  (92%). 
Raising  the  cutoff  score  by  five  points  (ST  =  97)  would  lead  to  slightly  higher  chances  of 
passing  for  average  male  and  female  Soldiers,  but  eligibility  would  be  reduced  by  1 1 


Table  2.  9 ID:  Probabi 
population)  will  pass  t 

lity  that  a  Regular  Anny  Soldier  (from  the  larger  Army  contract 
le  course  based  on  the  binary  logistic  model 

Cutoff  = 

82 

Cutoff = 

87 

Cutoff = 

92 

Cutoff = 

97 

Cutoff = 
102 

Percent  Eligible 
(Regular  Army) 

99.8% 

97.1% 

89.0% 

78.2% 

64.9%o 

Mean 

106.82 

107.67 

109.45 

111.59 

114.16 

Passing  Rates: 

High  School  Male 

81.1% 

82.4% 

84.8% 

87.3% 

89.8% 

College  Male 

High  School  Female 

College  Female 

percentage  points, 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
male  Regular  Army  Soldiers  by  education,  including  upper  and  lower  bounds  based  on 
the  standard  error  of  the  estimated  ST  coefficient.  For  a  particular  ST  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
corresponding  probability  of  passing.  For  example,  an  ST  score  of  100  corresponds  to  a 
passing  probability  of  about  70%  for  a  high  school  educated,  Regular  Army  male  Soldier. 
One  can  see  that  Soldiers  with  a  low  ST  score  stand  very  little  chance  of  passing  the 
course.  As  ST  scores  increase,  the  chance  that  a  Soldier  will  pass  the  course  increases 
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sharply.  In  addition,  the  graph  illustrates  that  Soldiers  with  some  college  education  have  a 
noticeably  better  chance  of  passing  this  course. 


Figure  1:  Predicted  Probalility  of  Passing:  Male  Soldiers 


—  Male  High  School  -  -  -  M/HS  Upper  -  -  -  M/HS  Lower  Male  College  —  -  M/C  Upper  —  -  M/C  Lower 


Figure  2  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
high  school  educated  Soldiers  by  gender,  including  upper  and  lower  bounds  based  upon 
the  standard  error  of  the  estimated  ST  coefficient.  One  can  see  that  Soldiers  with  a  low 
ST  score  stand  very  little  chance  of  passing  the  course.  As  ST  scores  increase,  the  chance 
that  a  Soldier  will  pass  the  course  increases  sharply.  In  addition,  the  graph  illustrates  that 
female  Soldiers  have  a  noticeably  better  chance  of  passing  this  course. 
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Figure  2:  Predicted  Probability  of  Passing:  High  School  Soldiers 


—  Male  High  School  —  -  M/HS  Upper  —  -  M/HS  Lower  Female  High  School  -  -  -  F/HS  Upper  —  -  F/HS  Lower 


Figure  3  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
Soldiers  by  education  and  gender.  The  graph  illustrates  that  women  with  some  college 
education  have  the  best  chance  at  passing  the  course,  while  high  school  educated  males 
have  the  lowest  chance  of  passing.  Figure  3  also  illustrates  the  magnitude  of  the  effect 
brought  about  by  gender  and  education.  At  the  current  cutoff  score  (ST  =  92)  college 
educated  female  Soldiers  have  more  than  a  40%  better  chance  of  passing  the  course 
compared  to  high  school  educated  males.  Even  at  a  ST  score  as  high  as  112,  college 
educated  women  still  have  approximately  a  10%  better  chance  of  passing  than  high 
school  educated  males. 
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Figure  3:  HS  vs.  College  &  Male  vs.  Female 
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ST  composite  score 


Appendix:  Soldier  Characteristics  (9 ID) 


Gender 

Male 

Female 

fail 

pass 

17.9% 

82.1% 

12.1% 

87.9% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

pass 

9.8% 

90.2% 

16.6% 

83.4% 

4.9% 

95.1% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

pass 

16.1% 

83.9% 

13.6% 

86.4% 

15.2% 

84.8% 
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9  ID  Risk  Analysis  of  Policy  Alternatives 


In  this  section  we  extend  the  policy  analysis  with  a  risk  analysis  of  the  policy 
alternatives  considered  using  simulation  methods.  We  employ  the  “@RISK”  software 
package  which  is  designed  as  an  add-in  for  Microsoft  Excel.44  There  are  four  steps  in  the 
risk  analysis. 

The  first  step  is  developing  a  model.  A  regression  model  for  predicting  the 
passing  rate  has  been  specified  and  estimated,  and  the  results  stored  in  Excel  worksheet 
format.  See  Table  1. 

The  second  step  is  identifying  the  uncertainty  inherent  in  both  the  input  variables 
(composite  scores,  Soldier  characteristics)  and  the  corresponding  estimated  parameters 
and  specifying  their  possible  values  with  probability  distributions. 

To  approximate  the  governing  composite  distribution,  we  use  the  larger  Army 
contract  population  (EAF  file:  those  who  contracted  from  Jan  1992  to  Aug  2003).  This 
distribution  can  be  assumed  to  follow  a  normal  distribution.  The  mean  and  standard 
deviation  estimated  from  the  contract  population  and  used  as  input  to  the  simulations  are 
shown  in  Table  R-l  (before  truncation). 


Table  R-l.  Input  infonnation  for  the  simulations 

Mean 

Std  error  / 
deviation 

Constant 

-9.009 

1.437 

ST  var 

107.12  (before  truncation) 

12.157 

ST  coeff 

.098 

.014 

GENDER  var 

52.7%  (female) 

GENDER  coeff 

.903 

.188 

GED  var 

5.7% 

GED  coeff 

.632 

.502 

COLLEGE  var 

11.5% 

COLLEGE  coeff 

1.255 

.438 

RESERVE  var 

48.4% 

RESERVE  coeff 

.169 

.189 

GUARD  var 

3.1% 

GUARD  coeff 

.104 

.527 

_ 
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To  approximate  the  distributions  for  the  demographic  variables  (GENDER,  GED, 
COLLEGE,  RESERVE,  and  GUARD)  we  rely  on  the  training  data  sample  used  to 
estimate  the  regression  model.  These  distributions  can  be  assumed  to  follow  binomial 
distributions  with  means  estimated  from  the  training  data  sample  as  shown  in  Table  R-l. 

To  approximate  the  distributions  for  the  estimated  parameters  in  the  logit  model, 
we  note  that,  under  very  general  conditions,  the  estimates  will  be  asymptotically  normally 
distributed ,45  The  estimated  coefficients  (means  and  standard  errors)  are  shown  in  Table 
R-l. 


The  third  step  is  analyzing  the  model  with  simulation  to  determine  the  range  and 
probabilities  of  all  possible  training  outcome  results.  The  results  reported  represent  the 
simulation  of  the  regression  prediction  equation  for  1000  iterations  using  a  fixed  seed.46 
The  comparisons  are  between  the  baseline  case  (ST  =  92)  and  the  policy  cases  (ST  =  87 
and  ST  =  97).  These  cases  are  delineated  by  the  cutoff  score  level  which  serves  as  the 
lower  truncation  point  in  the  governing  composite  input  distribution.  The  simulation 
results  in  Table  R-2  describe  the  effect  on  the  mean  probability  of  passing  as  the  ST 
cutoff  score  level  is  varied,  and  represent  an  improvement  of  the  policy  analysis 
presented  in  Table  2  because  they  reflect  the  simultaneous  effects  of  all  known  sources  of 
uncertainty. 


Table  R-2.  9 ID  simulation  resu 

Its 

ST  =87 

ST  =  92 

ST  =  97 

Mean  governing 
composite 

108.32 

109.55 

111.35 

Mean  probability 
of  passing 

73.6% 

75.2% 

77.8% 

Std  deviation 

28.8% 

27.9% 

26.1% 

The  frequency  distributions  of  simulated  effects  are  shown  below  in  the  histogram 
graphs.  The  histogram  portrays  the  full  range  of  outcomes  that  average  to  the  mean 
effect  as  reported.  Looking  at  the  histogram  for  the  baseline  case  (ST  =  92),  probability 
of  passing  is  plotted  on  the  horizontal  axis  and  the  height  of  each  bar  indicates  the 
relative  number  of  Soldiers  at  that  passing  probability.  The  information  in  the  histograms 
is  effectively  summarized  in  Table  R-3.  In  that  table  we  see  the  percentage  of  Soldiers 
that  meet  or  exceed  a  given  chance  of  passing.  For  example,  looking  at  the  baseline  case 
(ST  =  92),  about  68%  of  Soldiers  have  a  70%  or  greater  chance  of  passing. 


44  Software  is  available  from  Palisade  Corporation,  Newfield,  NY. 

45  The  intuition  is  that  logit  estimates  are  derived  from  maximum  likelihood  (ML)  estimation,  and  under 
fairly  general  conditions  ML  estimates  are  consistent  and  asymptotically  normally  distributed. 

46  Latin  Hypercube  (LH)  sampling  is  utilized  in  drawing  samples  for  the  input  distributions.  It  is  designed 
to  accurately  recreate  the  input  distribution  through  sampling  in  fewer  iterations  when  compared  with  the 
Monte  Carlo  method.  The  key  to  LH  sampling  is  stratification  of  the  input  probability  distributions. 
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Table  R-3. 

Percent  of  simulated  Soldiers  that  meet  or  exceed 
given  chance  of  passing 


Chance  of 
passing 

ST  =  87 

ST  =  92 

ST  =  97 

95.0% 

33.1% 

34.6% 

36.6% 

90.0% 

45.3% 

46.1% 

49.4% 

85.0% 

52.4% 

53.3% 

58.8% 

80.0% 

57.1% 

60.1% 

64.3% 

75.0% 

62.1% 

63.8% 

68.0% 

70.0% 

65.9% 

67.9% 

71.3% 

65.0% 

69.4% 

71.7% 

74.2% 

60.0% 

72.2% 

74.7% 

77.7% 

55.0% 

75.0% 

77.5% 

80.5% 

50.0% 

77.8% 

80.2% 

83.5% 

We  find  that  the  mean  probability  of  passing  in  the  simulation  is  75%  (Table  R-2, 
baseline).  Looking  at  the  histograms,  we  see  the  largest  portion  of  Soldiers  in  the  highest 
range  of  passing  scores,  and  the  remainder  distributed  in  a  stair  step  into  the  lower  range. 
We  note  that  the  mean  simulated  passing  probability  is  10-20  percentage  points  lower 
than  that  for  any  of  the  means  produced  by  the  static  prediction  at  the  mean  ST  score 
(Table  2).  Sensitivity  analysis  indicates  that  this  is  due  to  the  standard  error  for  the  ST 
coefficient,  which  brings  about  a  relatively  large  standard  deviation  in  the  predicted 
passing  probability.  Table  R-2  shows  that  in  any  of  the  conditions,  one  standard  deviation 
greater  than  the  mean  is  over  100%  passing  probability,  so  in  effect  the  average 
probability  of  passing  is  censored  at  that  point,  negatively  skewing  the  data.  This  skew 
will  produce  a  mean  that  is  considerably  lower  than  a  mean  that  would  be  calculated  if 
error  were  not  taken  into  account.  In  the  simulated  predictions  we  are  calculating  an 
average  over  all  Soldiers  and  taking  into  account  all  sources  of  uncertainty,  whereas  in 
the  static  prediction  we  are  calculating  the  probability  of  passing  only  at  the  mean  ST 
score,  ignoring  the  distribution  of  Soldier  scores  etc.  as  well  as  uncertainty.  While 
recognizing  this  variability  is  important,  we  also  note  that  the  policy  case  impacts  relative 
to  the  baseline  are  the  same  in  both  simulation  and  static  prediction  modes. 


The  fourth  step  is  making  a  decision  based  on  the  results  provided  and  the 
preferences  of  the  school  proponent.  This  involves  the  evaluation  by  the  school 
proponent  of  the  tradeoff  between  increased  eligibility  and  reduced  passing  probabilities 
(and  vice-versa). 
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Distribution  for  PASS  EQUATION  97 


X<=0.13  X<=1 


Distribution  for  PASS  EQUATION  92 


X<=0.B  X<=1 
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Distribution  for  PASS  EQUATION  87 


X<=0.13  X  <=1 
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91E:  Dental  Specialist 


The  final  sample  included  1 143  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (93%)  or  failed  for  academic  reasons  (7%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  female  (55%,  45%  male),  most  had  a  high  school 
diploma  but  not  more  education  (86%,  10%  some  college,  4%  GED  or  less),  and  most 
were  from  the  Regular  Army  (56%,  33%  Reserve,  1 1%  National  Guard).  The  governing 
AA  composite,  Skilled  Technical  (ST),  for  this  MOS  has  a  cutoff  score  of  92;  the  sample 
mean  is  102.79  (standard  deviation  =  9.054).  See  the  Appendix  for  a  description  of 
Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 


Table  1.  9  IE:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

78.587*** 

Log  Likelihood 

521.662 

Nagelkerke  R  Square 

.163 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-12.068 

.000 

30.364*** 

GED 

-.877 

.416 

3.426 

COLLEGE 

-.183 

.833 

.200 

RESERVE 

.143 

1.154 

.303 

GUARD 

.679 

1.971 

1.785 

GENDER 

1.197 

3.312 

22.667*** 

ST 

.140 

1.150 

40.138*** 

*  =  p  <  .05  ***=p<.001 

**  =  p<  .01 
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Table  1  and  indicate  that  a  model,  including  the  ST  composite  and  gender  (GENDER  = 
female),  accounts  for  about  16%  of  the  variation  in  the  dependent  variable.  This  model  is 
statistically  significant  (y2  =  78.587,  P  <  .001),  and  has  moderate  explanatory  power 
(Nagelkerke  R2  =  .163). 

There  is  a  statistically  significant  effect  for  the  ST  composite.  The  model 
suggests  that  having  a  higher  ST  composite  score  increases  the  odds  of  passing.  At  the 
mean  ST  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  15%  in 
the  odds  of  passing  the  course,  and  a  five-point  increase  in  ST  would  increase  the  odds  of 
passing  by  101%.  There  are  also  statistically  significant  effects  for  GENDER,  with 
increased  odds  of  passing  the  course  at  231%  (for  females). 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

Q(f>’x)  /  (1  + 

The  following  analysis  is  focused  upon  male  and  female  Soldiers  with  a  high  school 
diploma  or  some  college  education  from  the  Regular  Army.  Based  on  the  model,  when 
the  cutoff  score  is  at  its  current  level  (ST  =  92),  male  Soldiers  with  a  high  school  diploma 
and  an  average  ST  score  (ST  =  109.45)  have  approximately  a  96%  chance  of  passing; 
female  Soldiers  would  have  a  slightly  higher  chance  of  passing  (99%).  Currently,  about 
89%  of  Soldiers  are  eligible  for  MOS  9  IE  assignment  at  the  current  cutoff  (ST  =  92). 
Lowering  the  cutoff  by  five  points  (ST  =  87)  would  increase  eligibility  by  nine 
percentage  points  (to  97%)  and  the  average  male  Soldier  who  would  qualify  for  training 
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would  have  about  the  same  chance  of  passing  (95%),  as  would  the  average  female 
Soldier  (99%).  Raising  the  cutoff  score  by  five  points  (ST  =  97),  the  average  male  and 
female  Soldier  would  have  about  the  same  chance  of  passing  (97%,  99%)  but  eligibility 
would  fall  by  about  1 1  percentage  points. 


Table  2.  9 IE:  Probabi 
population)  will  pass  t 

lity  that  a  Regular  Army  Soldier  (from  the  larger  Army  contract 
le  course  based  on  the  binary  logistic  model 

Cutoff  = 

82 

Cutoff = 

87 

Cutoff = 

92 

Cutoff = 

97 

Cutoff = 
102 

Percent  Eligible 
(Regular  Army) 

99.8% 

97.1% 

89.0% 

78.2% 

64.9% 

Mean 

106.82 

107.67 

109.45 

111.59 

114.16 

Passing  Rates: 

High  School  Male 

msm- 

mm i 

!SK 

mmm 

High  School  Female 

mmm\ 

— 

■B 

■B 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
male  and  female  Regular  Army  Soldiers  with  a  high  school  education,  including  upper 
and  lower  bounds  based  upon  the  standard  error  of  the  estimated  ST  coefficient.  For  a 
particular  ST  score,  trace  a  vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to 
the  axis  to  determine  the  corresponding  probability  of  passing.  For  example,  an  ST  score 
of  100  corresponds  to  a  passing  probability  of  about  87%  for  a  high  school  educated, 
Regular  Anny  male  Soldier.  One  can  see  that  Soldiers  with  a  low  ST  score  stand  very 
little  chance  of  passing  the  course.  As  ST  scores  increase,  the  chance  that  a  Soldier  will 
pass  the  course  increases  sharply.  In  addition,  the  graph  illustrates  that  female  Soldiers 
have  a  noticeably  better  chance  of  passing  this  course  at  lower  scores. 
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Chi 


Figure  1:  Predicted  Probability  of  Passing:  High  School  Soldiers 


91G:  Patient  Administration  Specialist 


The  final  sample  included  652  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (97%)  or  failed  for  academic  reasons  (3%)  were 
included  in  the  analysis  sample  of  652  Soldiers  because  we  assume  that  non-completion 
for  non-academic  reasons  is  not  related  to  cognitive  criteria. 

In  this  sample,  most  were  female  (54%,  46%  male),  most  had  a  high  school 
diploma  but  not  more  education  (82%,  13%  some  college,  5%  GED  or  less),  and  most 
were  from  the  Anny  Reserve  (53%,  27%  Regular  Anny,  20%  National  Guard).  The 
governing  AA  composite,  Clerical  (CL),  for  this  MOS  has  a  cutoff  score  of  92;  the 
sample  mean  is  104.06  (standard  deviation  =  10.509).  See  the  Appendix  for  a  description 
of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 


Table  1.  91G:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

8.052* 

Log  Likelihood 

184.382 

Nagelkerke  R  Square 

.04 

t8 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-.574 

.564 

.125 

RESERVE 

-.258 

.773 

.318 

GENDER 

.751 

2.120 

2.686 

CL 

.037 

1.037 

5.261* 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 
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Table  1  and  indicate  that  a  model,  including  the  CL  composite,  component,  and  gender 
(GENDER  =  female),  accounts  for  about  five  percent  of  the  variation  in  the  dependent 
variable.  This  model  is  statistically  significant  (x  =  8.052,  P  =  .048),  and  has  limited 
explanatory  power  (Nagelkerke  R“  =  .048). 

There  is  a  statistically  significant  effect  for  the  CL  composite.  The  model 
suggests  that  having  a  higher  CL  composite  score  increases  the  odds  of  passing.  At  the 
mean  CL  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  4%  in  the 
odds  of  passing  the  course,  and  a  five-point  increase  in  CL  would  increase  the  odds  of 
passing  by  20%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

Q(P’x)  /  (1  +  Q(fr*)y 

The  following  analysis  is  focused  upon  female  Soldiers  with  a  high  school  diploma  (the 
modal  categories)  from  the  Regular  Army.  Based  on  the  model,  when  the  cutoff  score  is 
at  its  current  level  (CL  =  92),  female  Soldiers  with  a  high  school  diploma  and  an  average 
CL  score  (CL  =  109.17)  have  approximately  a  94%  chance  of  passing.  Currently,  about 
92%  of  Soldiers  are  eligible  for  MOS  91G  assignment  at  the  current  cutoff  (CL  =  92). 
Lowering  the  cutoff  by  five  points  (CL  =  87)  would  increase  eligibility  by  seven 
percentage  points  (to  99%)  and  the  average  female  Soldier  who  would  qualify  for 
training  would  have  about  the  same  chance  of  passing  (93%).  Raising  the  cutoff  score 
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by  five  points  (CL  =  97),  the  average  female  Soldier  would  have  about  the  same  chance 
of  passing  (94%)  but  eligibility  would  fall  by  about  12  percentage  points. 


Table  2.  91G:  Probabi 
population)  will  pass  t 

lity  that  a  Regular  Army  Soldier  (from  the  larger  Army  contract 
re  course  based  on  the  binary  logistic  model 

Cutoff  = 

82 

Cutoff = 

87 

Cutoff = 

92 

Cutoff = 

97 

Cutoff = 
102 

Percent  Eligible 
( Regular  Army) 

99.9% 

99.0% 

91.6% 

80.0% 

67.6% 

Mean 

107.49 

107.69 

109.17 

111.37 

113.62 

Passing  Rates: 

High  School  Female 

93.4% 

93.5% 

93.8% 

94.2% 

94.7% 

Figure  1  shows  the  relationship  between  CL  and  the  probability  of  passing  for 
female  Regular  Army  Soldiers,  including  upper  and  lower  bounds  based  upon  the 
standard  error  of  the  estimated  CL  coefficient.  For  a  particular  CL  score,  trace  a  vertical 
line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  detennine  the 
corresponding  probability  of  passing.  For  example,  a  CL  score  of  100  corresponds  to  a 
passing  probability  of  about  90%  for  a  high  school  educated,  Regular  Army  female 
Soldier.  One  can  see  that  female  Soldiers  with  a  low  CL  score  still  have  a  moderately 
good  chance  of  passing  the  course.  As  CL  scores  increase,  the  chance  that  a  Regular 
Army  female  Soldier  will  pass  the  course  increases. 
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Chi 


Figure  1:  Probability  of  Passing:  Female  Regular  Army  Soldiers 


Appendix:  Soldier  Characteristics  (91G) 


Gender 

Outcome 

Male 

Female  | 

fail 

4.6% 

2.3% 

pass 

95.4% 

97.7% 

Education  Level  I 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

6.1% 

3.8% 

.0% 

pass 

93.9% 

96.2% 

100.0% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

3.8% 

1 .5% 

pass 

96.2% 

98.5% 
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91H:  Optical  Laboratory  Specialist 


The  final  sample  included  86  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (88.4%)  or  failed  for  academic  reasons  (1 1.6%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  of  the  Soldiers  in  MOS  91H  were  male  (60%,  40%  female), 
most  had  a  high  school  diploma  but  not  more  education  (84%,  13%  some  college,  4% 
GED  or  less),  and  most  were  from  Regular  Anny  (77%,  9%  National  Guard,  14%  Anny 
Reserve).  The  governing  AA  composite,  General  Maintenance  (GM),  for  this  MOS  has  a 
cutoff  score  of  97;  the  sample  mean  is  105.99  (standard  deviation  =  8.823).  See  the 
Appendix  for  a  description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  a  model,  including  the  GM  composite  and  gender  (GENDER  = 
female),  accounts  for  almost  12  percent  of  the  variation  in  the  dependent  variable.  This 


Table  1.  91H:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

5.235 

Log  Likelihood 

56.590 

Nagelkerke  R  Square 

.115 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-12.128 

.000 

2.406 

GENDER 

.068 

1.071 

.010 

GM 

.137 

1.147 

3.202 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .005 
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2 

model  is  borderline  statistically  significant  (y  =  5.235,  P  =  .073),  and  has  moderate 
explanatory  power  (Nagelkerke  R“  =  .  1 15). 

There  is  no  statistically  significant  effect  for  the  GM  composite.  The  model 
suggests  that  the  GM  composite  score  does  not  predict  increased  odds  of  passing,  but 
there  are  too  few  failure  observations  for  a  definitive  conclusion. 

Policy  Analysis.  Given  the  model’s  lack  of  significance,  this  model  is  not 
appropriate  for  conducting  policy  analysis. 

Excursion.  In  view  of  the  fact  that  ST  is  the  governing  composite  for  many  of  the 
other  technical  MOS  within  CMF  91,  we  also  estimated  a  regression  model  using  ST  in 
place  of  the  GM  composite.  The  results  in  Table  2  indicate  a  statistically  significant 
model,  and  moderate  explanatory  power  (Nagelkerke  R"  =  .229).  There  is  also  a 
statistically  significant  effect  for  the  ST  composite.  The  model  suggests  that  a  higher  ST 
composite  score  increases  the  odds  of  passing,  but  there  are  too  few  failure  observations 
for  a  definitive  conclusion. 


Table  2.  91H:  Results  of  the  binary  logistic 
prediction  model  utilizing  ST  composite 


Chi-  Square 

10.718** 

Log  Likelihood 

51.107 

Nagelkerke  R  Square 

.229 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-21.622 

.000 

5.780* 

GENDER 

-.104 

.902 

.020 

ST 

.227 

1.255 

6.673** 

*  =  p  <  .05  ***=p<.001 

**  =p  <  .01 
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Appendix:  Soldier  Characteristics  (91H) 


Gender 

Outcome 

Male 

Female  | 

fail 

9.8% 

14.3% 

pass 

90.2% 

85.7% 

Education  Level  j 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

.0% 

12.5% 

9.1% 

pass 

100.0% 

87.5% 

90.9% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

10.6% 

8.3% 

25.0% 

89.4% 

91.7% 

75.0% 
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91J:  Medical  Logistics  Specialist 


The  final  sample  included  709  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (89%)  or  failed  for  academic  reasons  (11%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  of  the  Soldier  in  MOS  91 J  were  male  (56%,  44%  female) 
most  had  a  high  school  diploma  but  not  more  education  (84%,  10%  some  college,  6% 
GED  or  less),  and  most  were  from  Regular  Anny  (49%,  40%  National  Guard,  11%  Anny 
Reserve).  The  governing  AA  composite,  Clerical  (CL),  for  this  MOS  has  a  cutoff  score  of 
92;  the  sample  mean  is  102.92  (standard  deviation  =  1 1.858).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 


Table  1.  91J:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

31.550*** 

Log  Likelihood 

459.984 

Nagelkerke  R  Square 

.087 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-5.856 

.005 

10.525*** 

GED 

-.560 

.571 

1.114 

GUARD 

.661 

1.938 

1.700 

RESERVE 

.404 

1.497 

1.977 

GENDER 

.416 

1.516 

2.554 

CL 

.069 

1.072 

18.589*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 
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Table  1  and  indicate  that  a  model,  including  the  CL  composite,  GED  (GED  or  less 
education),  GUARD  (National  Guard  Membership),  RESERVE  (Army  Reserve 
Membership)  and  gender  (GENDER  =  female),  accounts  for  almost  nine  percent  of  the 
variation  in  the  dependent  variable.  This  model  is  statistically  significant  (x  =  31.550,  P 
<  .001),  and  has  moderate  explanatory  power  (Nagelkerke  R2  =  .087). 

There  is  a  statistically  significant  effect  for  the  CL  composite.  The  model 
suggests  that  having  a  higher  CL  composite  score  increases  the  odds  of  passing.  At  the 
mean  CL  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  7%  in  the 
odds  of  passing  the  course,  and  a  five-point  increase  in  CL  would  increase  the  odds  of 
passing  by  41%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

qw'x)  /  (1  +  q(P’x>). 

The  following  analysis  is  focused  upon  male  Soldiers  with  a  high  school  diploma  (the 
modal  categories)  from  the  Regular  Army.  Based  on  the  model,  when  the  cutoff  score  is 
at  its  current  level  (CL  =  92),  male  Soldiers  with  a  high  school  diploma  and  an  average 
CL  score  (CL  =  109.17)  have  approximately  a  91%  chance  of  passing.  Currently,  about 
92%  of  Soldiers  are  eligible  for  MOS  91 J  assignment  at  the  current  cutoff  (CL  =  92). 
Lowering  the  cutoff  by  five  points  (CL  =  87)  would  increase  eligibility  by  seven 
percentage  points  (to  99%)  and  the  average  male  Soldier  who  would  qualify  for  training 
would  have  about  the  same  chance  of  passing  (90%).  Raising  the  cutoff  score  by  five 
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points  (CL  =  97),  the  average  male  Soldier  would  have  about  the  same  chance  of  passing 
(92%)  while  eligibility  would  fall  by  about  12  percentage  points. 


Table  2.  91J:  Probability  that  a  Soldier  (from  the  larger  Anny  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model 

Cutoff  = 

82 

Cutoff = 

87 

Cutoff = 

92 

Cutoff = 

97 

Cutoff = 
102 

Percent  Eligible 
(Regular  Army) 

99.9% 

99.0% 

91.6% 

80.0% 

67.6% 

Mean 

107.49 

107.69 

109.17 

111.37 

113.62 

Passing  Rates: 

Male 

89.7% 

89.8% 

90.7% 

91.9% 

93.0% 

Figure  1  shows  the  relationship  between  CL  and  the  probability  of  passing  for 
male  Regular  Army  Soldiers,  including  upper  and  lower  bounds  based  upon  the  standard 
error  of  the  estimated  CL  coefficient.  For  a  particular  CL  score,  trace  a  vertical  line  up  to 
the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the  corresponding 
probability  of  passing.  For  example,  a  CL  score  of  100  corresponds  to  a  passing 
probability  of  about  85%  for  a  high  school  educated,  Regular  Army  male  Soldier.  One 
can  see  that  male  Soldiers  with  a  low  CL  score  have  a  relatively  low  chance  of  passing 
the  course.  As  CL  scores  increase,  the  chance  that  a  Regular  Army  male  Soldier  will  pass 
the  course  increases  somewhat  quickly. 
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y  of  pasing 


Figure  1:  Predicted  Probability  of  Passing:  Male,  Regular  Army 


Appendix:  Soldier  Characteristics  (91J) 


Gender 

Male 

Female 

Outcome 

fail 

11.2% 

10.8% 

pass 

88.8% 

89.2% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

pass 

12.0% 

88.0% 

2.7% 

97.3% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

pass 

13.6% 

86.4% 

8.9% 

91.1% 

6.7% 

93.3% 
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91K:  Medical  Laboratory  Specialist 


The  final  sample  included  123  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (81%)  or  failed  for  academic  reasons  (19%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  of  the  Soldier  in  MOS  9  IK  were  male  (55%,  45%  female) 
most  had  a  high  school  diploma  but  not  more  education  (71%,  29%  some  college,  1% 
GED  or  less),  and  most  were  from  Regular  Anny  (76%,  4%  National  Guard,  20%  Anny 
Reserve).  The  governing  AA  composite,  Skilled  Technical  (ST),  for  this  MOS  has  a 
cutoff  score  of  107;  the  sample  mean  is  1 17.72  (standard  deviation  =  7.500).  See  the 
Appendix  for  a  description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 


Table  1.  9  IK:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

16.614** 

Log  Likelihood 

101.614 

Nagelkerke  R  Square 

.204 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-14.626 

.000 

8.183** 

COLLEGE 

-.244 

.784 

.196 

RESERVE 

-.900 

.407 

2.417 

GENDER 

1.406 

4.082 

6.024* 

ST 

.136 

1.146 

9.602** 

*  =  p  <  .05  ***=p<.001 

**  =p  <  .01 
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Table  1  and  indicate  that  a  model,  including  the  ST  composite,  COLLEGE  (some  college 
education),  RESERVE  (Army  Reserve  Membership)  and  gender  (GENDER  =  female), 
accounts  for  20%  of  the  variation  in  the  dependent  variable.  This  model  is  statistically 
significant  (y  =  16.614,  P  =  .002),  and  has  moderate  explanatory  power  (Nagelkerke  R“ 

=  .204). 

There  is  a  statistically  significant  effect  for  the  ST  composite.  The  model 
suggests  that  having  a  higher  ST  composite  score  increases  the  odds  of  passing.  At  the 
mean  ST  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  15%  in 
the  odds  of  passing  the  course,  and  a  five-point  increase  in  ST  would  increase  the  odds  of 
passing  by  97%.  There  is  also  a  statistically  significant  effect  for  GENDER  where  the 
odds  of  a  female  Soldier  passing  the  course  are  308%  higher  than  the  odds  for  a  male 
Soldier. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

q(P’x)  /  (1  +  e(fl’x>). 

The  following  analysis  is  focused  upon  male  Soldiers  with  a  high  school  diploma  (the 
modal  categories)  from  the  Regular  Army.  Based  on  the  model,  when  the  cutoff  score  is 
at  its  current  level  (ST  =  107),  male  Soldiers  with  a  high  school  diploma  and  an  average 
ST  score  (ST  =  1 16.55)  have  approximately  a  77%  chance  of  passing,  while  their  female 
counterparts  have  approximately  a  93%  chance  of  passing.  Currently,  about  50%  of 
Soldiers  are  eligible  for  MOS  91K  assignment  at  the  current  cutoff  (ST  =  107).  Lowering 
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the  cutoff  by  five  points  (ST  =  102)  would  increase  eligibility  by  15  percentage  points  (to 
65%)  and  the  average  male  Soldier  who  would  qualify  for  training  would  have  a 
somewhat  lower  chance  of  passing  (70%),  while  the  average  female  soldier  would  have 
almost  the  same  chance  of  passing  (90%)  as  at  the  current  cutoff.  In  raising  the  cutoff 
score  by  five  points  (ST  =  112),  the  average  qualifying  male  Soldiers  would  have  a 


Table  2.  9 IK:  Probabi 
population)  will  pass  t 

lity  that  a  Regular  Army  Soldier  (from  the  larger  Army  contract 
re  course  based  on  the  binary  logistic  model 

Cutoff  = 

97 

Cutoff = 
102 

Cutoff = 
107 

Cutoff = 
112 

Cutoff = 
117 

Percent  Eligible 
(Regular  Army) 

78.2% 

64.9%o 

49.6% 

34.7%, 

22.0% 

Mean 

111.14 

113.6%o 

116.55 

119.83 

123.23 

Passing  Rates: 

Male 

62.0% 

69.5% 

77.3% 

84.2% 

89.4% 

Female 

86.9% 

90.3% 

93.3% 

95.6% 

97.2% 

higher  chance  of  passing  (84%)  and  the  average  female  Soldier  only  a  slightly  higher 
chance  of  passing  (96%),  while  eligibility  would  be  reduced  by  15  percentage  points. 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
male  and  female  Regular  Army  Soldiers,  including  upper  and  lower  bounds  based  upon 
the  standard  error  of  the  estimated  ST  coefficient.  For  a  particular  ST  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
corresponding  probability  of  passing.  For  example,  a  ST  score  of  100  corresponds  to  a 
passing  probability  of  about  70%  for  a  high  school  educated,  Regular  Army  female 
Soldier.  One  can  see  that  male  Soldiers  with  a  low  ST  score  have  very  little  chance  of 
passing,  while  female  soldier  with  low  ST  scores  have  a  somewhat  higher  chance  of 
passing.  As  ST  scores  increase,  the  chance  that  a  Regular  Army  male  Soldier  will  pass 
the  course  increases  more  quickly  for  female  Soldiers  than  for  male  Soldiers. 
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Chance  of  Passing 


Figure  1 :  Predicted  Probability  of  Passing:  Regular  Army  Soldiers  with  a  High  School  Diploma 
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Appendix:  Soldier  Characteristics  (9 IK) 


Gender 

Male 

Female 

Outcome 

fail 

HU 

12.5% 

pass 

■EH 

87.5% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

pass 

100.0% 

.0% 

17.2% 

82.8% 

20.0% 

80.0% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

pass 

17.0% 

83.0% 

29.2% 

70.8% 

.0% 

100.0% 
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91M:  Hospital  Food  Service  Specialist 


The  final  sample  included  837  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (98.7%)  or  failed  for  academic  reasons  (1.3%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (64%,  36%  female),  most  had  a  high  school 
diploma  but  not  more  education  (88%,  8%  some  college,  4%  GED  or  less),  and  most 
were  from  Army  Reserve  (69%,  31%  Regular  Anny,  <1%  National  Guard).  The 
governing  AA  composite,  Operator/Food  (OF),  for  this  MOS  has  a  cutoff  score  of  95;  the 
sample  mean  is  105.20  (standard  deviation  =  7.918).  See  the  Appendix  for  a  description 
of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  a  model,  including  the  OF  composite,  Anny  component 


Table  1.  91M:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

5.177 

Log  Likelihood 

111.980 

Nagelkerke  R  Square 

.047 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

1.989 

7.311 

.185 

RESERVE 

1.347 

3.845 

4.269* 

GENDER 

.213 

1.237 

.107 

OF 

.015 

1.015 

.109 

*  =  p  <  .05  ***  =  p  <  .001 
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(RESERVE  =  Army  Reserve)  and  gender  (GENDER  =  female),  accounts  for  five  percent 
of  the  variation  in  the  dependent  variable.  This  model  is  not  statistically  significant  (x  = 
5.177,  P  =  .159),  and  has  limited  explanatory  power  (Nagelkerke  R  =  .047). 

There  is  no  statistically  significant  effect  for  the  OF  composite,  though  there  is 
one  for  Army  Reserve  membership.  The  model  suggests  that  the  OF  composite  score 
does  not  predict  increased  odds  of  passing,  but  there  are  too  few  failure  observations  for  a 
definitive  conclusion. 

Policy  Analysis.  Given  the  model’s  lack  of  explanatory  power,  and 
notwithstanding  the  statistically  significant  effect  for  Reserve  membership,  this  model  is 
not  appropriate  for  conducting  policy  analysis. 
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Appendix:  Soldier  Characteristics  (91M) 


Gender 

Outcome 

Male 

Female  [ 

fail 

9.0% 

6.2% 

pass 

91.0% 

93.8% 

Education  Level  I 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

3.8% 

9.5% 

3.0% 

pass 

96.2% 

90.5% 

97.0% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

8.6% 

6.3% 

8.8% 

pass 

91.4% 

93.7% 

91.3% 
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91P:  Radiology  Specialist 


The  final  sample  included  814  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  the  first  phase  training  for  this  MOS 
during  a  period  from  2001  to  2004.  Only  Soldiers  who  graduated  (92%)  or  failed  for 
academic  reasons  (8%)  were  included  in  the  analysis  sample  of  814  Soldiers  because  we 
assume  that  non-completion  for  non-academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (65%,  35%  female),  most  had  a  high  school 
diploma  but  not  more  education  (76%,  21%  some  college,  3%  GED  or  less),  and  most 
were  from  were  from  the  Regular  Anny  (63%,  10%  National  Guard,  27%  Reserve).  The 
governing  AA  composite,  Skilled  Technical  (ST),  for  this  MOS  has  a  cutoff  score  of  107; 
the  sample  mean  is  112.11  (standard  deviation  =  8.648).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  model,  including  the  ST  composite,  education  level 


Table  1.  9  IP:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

28.356*** 

Log  Likelihood 

424.894 

Nagelkerke  R  Square 

.080 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-7.699 

.000 

9.082** 

COLLEGE 

1.022 

2.778 

4.529* 

RESERVE 

.278 

1.321 

.741 

GUARD 

.085 

1.089 

.038 

GENDER 

.586 

1.797 

4.008* 

ST 

.085 

1.089 

14.300*** 

*  =  p  <  .05  ***  =  p<.001 

**  =  p  <  .01 
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(COLLEGE  =  some  college),  Army  component  (GUARD  =  National  Guard;  RESERVE 
=  Army  Reserve)  and  gender  (GENDER  =  female),  accounts  for  about  eight  percent  of 
the  variation  in  the  dependent  variable.  This  model  is  statistically  significant  (y  =  28.356, 
P  <  .001),  and  has  moderate  explanatory  power  (Nagelkerke  R  =  .080). 

There  is  a  statistically  significant  effect  for  the  ST  composite.  The  model 
suggests  that  having  a  higher  ST  composite  score  increases  the  odds  of  passing.  At  the 
mean  ST  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  9%  in  the 
odds  of  passing  the  course,  and  a  five-point  increase  in  ST  would  increase  the  odds  of 
passing  by  53%.  There  are  also  statistically  significant  effects  for  COLLEGE  and 
GENDER,  with  increased  odds  of  passing  the  course  at  178%  and  78%  (for  females), 
respectively. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

Q(P’X>  /  (1  + 

The  following  analysis  is  focused  upon  male  and  female  Soldiers  with  a  high  school 
diploma  or  some  college  education  from  the  Regular  Army.  Based  on  the  model,  when 
the  cutoff  score  is  at  its  current  level  (ST  =  107),  male  Soldiers  with  a  high  school 
diploma  (the  modal  categories)  and  an  average  ST  score  (ST  =  1 16.55)  have 
approximately  a  90%  chance  of  passing.  Currently,  about  50%  of  Soldiers  are  eligible  for 
MOS  91P  assignment  at  the  current  cutoff  (ST  =  107).  Lowering  the  cutoff  by  five 
points  (ST  =  102)  would  increase  eligibility  by  15  percentage  points  (to  65%),  and  the 


294 


average  male  Soldier  who  would  qualify  for  training  would  have  a  somewhat  lower 
chance  of  passing  (88%).  In  raising  the  cutoff  score  by  five  points  (ST  =  1 12),  the 
average  male  Soldier  would  have  a  slightly  higher  chance  of  passing  (92%)  while 


Table  2.  9 IP:  Probabi 
population)  will  pass  t 

Lity  that  a  Regular  Anny  Soldier  (from  the  larger  Army  contract 
le  course  based  on  the  binary  logistic  model 

Cutoff  = 

97 

Cutoff = 
102 

Cutoff = 
107 

Cutoff = 
112 

Cutoff = 
117 

Percent  Eligible 
(Regular  Army) 

78.2% 

64.9%> 

49.6%, 

34. 7% 

22.0% 

Mean 

111.14 

113.6 

116.55 

119.83 

123.23 

Passing  Rates: 

High  School  Male 

85.2% 

87.6% 

90.1% 

92.3% 

94.1% 

College  Male 

— - 

High  School  Female 

wesm- 

rnsEmm 

College  Female 

m mm-. 

eligibility  would  be  reduced  by  15  percentage  points. 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
Regular  Anny  male  Soldiers  by  education,  including  upper  and  lower  bounds  based  on 
the  standard  error  of  the  estimated  ST  coefficient.  For  a  particular  ST  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
corresponding  probability  of  passing.  For  example,  an  ST  score  of  100  corresponds  to  a 
passing  probability  of  about  70%  for  a  high  school  educated,  Regular  Army  male  Soldier. 
One  can  see  that  Soldiers  with  a  low  ST  score  stand  very  little  chance  of  passing  the 
course.  As  ST  scores  increase,  the  chance  that  a  Soldier  will  pass  the  course  increases 
sharply.  In  addition,  the  graph  illustrates  that  Soldiers  with  some  college  education  have  a 
noticeably  better  chance  of  passing  this  course. 
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Figure  1:  Predicted  Probalility  of  Passing:  Male  Soldiers 


—  High  School  -  -  -  HS  Upper  -  -  -  HS  Lower  College  —  -  Col  Upper  —  -  Col  Lower 


Figure  2  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
high  school  educated  Soldiers  by  gender,  including  upper  and  lower  bounds  based  upon 
the  standard  error  of  the  estimated  ST  coefficient.  One  can  see  that  Soldiers  with  a  low 
ST  score  stand  very  little  chance  of  passing  the  course.  As  ST  scores  increase,  the  chance 
that  a  Soldier  will  pass  the  course  increases  sharply.  In  addition,  the  graph  illustrates  that 
female  Soldiers  have  a  noticeably  better  chance  of  passing  this  course. 
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Figure  2:  Predicted  Probalility  of  Passing:  High  School  Soldiers 


■'  Male  -  -  -  Male  Upper  -  -  -  Male  Lower  Female  —  ~  Female  Upper  —  ■  Female  Lower 


Figure  3  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
Soldiers  by  education  and  gender.  The  graph  illustrates  that  women  with  some  college 
education  have  the  best  chance  at  passing  the  course,  while  high  school  educated  males, 
the  modal  students,  have  the  lowest  chance  of  passing.  Figure  3  also  illustrates  the 
magnitude  of  the  effect  brought  about  by  gender  and  education.  At  the  current  cutoff 
score  (ST  =  102)  college  educated  female  Soldiers  have  more  than  a  20  percentage  point 
better  chance  of  passing  the  course  compared  to  high  school  educated  males.  Even  at  a 
ST  score  as  high  as  120,  college  educated  women  still  have  approximately  a  5  percentage 
point  better  chance  of  passing  than  high  school  educated  males. 
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Appendix:  Soldier  Characteristics  (9 IP) 


Gender 

Male 

Female 

Outcome 

fail 

HU 

12.5% 

pass 

■EH 

87.5% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

pass 

100.0% 

.0% 

17.2% 

82.8% 

20.0% 

80.0% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

pass 

17.0% 

83.0% 

29.2% 

70.8% 

.0% 

100.0% 
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91Q:  Pharmacy  Specialist 


The  final  sample  included  572  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (77%)  or  failed  for  academic  reasons  (23%)  were 
included  in  the  analysis  sample  of  572  Soldiers  because  we  assume  that  non-completion 
for  non-academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (55%,  45%  female),  most  had  a  high  school 
diploma  but  not  more  education  (82%,  17%  some  college,  1%  GED  or  less),  and  most 
were  from  were  from  the  Regular  Anny  (67%,  4%  National  Guard,  29%  Reserve).  The 
governing  AA  composite,  Skilled  Technical  (ST),  for  this  MOS  has  a  cutoff  score  of  92; 
the  sample  mean  is  107.66  (standard  deviation  =  10.338).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 


Table  1.  91Q:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

64.255*** 

Log  Likelihood 

546.426 

Nagelkerke  R  Square 

.162 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-8.136 

.000 

33.200*** 

COLLEGE 

.715 

2.045 

4.091* 

RESERVE 

.219 

1.245 

.804 

GUARD 

.093 

1.097 

.028 

GENDER 

.819 

2.268 

13.669*** 

ST 

.084 

1.087 

39.512*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 
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Table  1  and  indicate  that  a  model,  including  the  ST  composite,  education  level 
(COLLEGE  =  some  college),  Anny  component  (GUARD  =  National  Guard;  RESERVE 
=  Army  Reserve)  and  gender  (GENDER  =  female),  accounts  for  about  16%  of  the 
variation  in  the  dependent  variable.  This  model  is  statistically  significant  (x  =  64.255,  P 
<  .001),  and  has  moderate  explanatory  power  (Nagelkerke  R2  =  .162). 

There  is  a  statistically  significant  effect  for  the  ST  composite.  The  model 
suggests  that  having  a  higher  ST  composite  score  increases  the  odds  of  passing.  At  the 
mean  ST  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  9%  in  the 
odds  of  passing  the  course,  and  a  five-point  increase  in  ST  would  increase  the  odds  of 
passing  by  53%.  There  are  also  statistically  significant  effects  for  COLLEGE  and 
GENDER,  with  increased  odds  of  passing  the  course  at  105%  and  127%  (for  females), 
respectively. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  applicant  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

q(P’x)  /  (1  +  e(p’x}). 

The  following  analysis  is  focused  upon  male  and  female  Soldiers  with  a  high  school 
diploma  or  some  college  education  from  the  Regular  Army.  Based  on  the  model,  when 
the  cutoff  score  is  at  its  current  level  (ST  =  92),  male  Soldiers  with  a  high  school  diploma 
(the  modal  categories)  and  an  average  ST  score  (ST  =  109.06)  have  approximately  a  74% 
chance  of  passing.  Currently,  about  89%  of  Soldiers  are  eligible  for  MOS  91Q 
assignment  at  the  current  cutoff  (ST  =  92).  Lowering  the  cutoff  by  five  points  (ST  =  87) 
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would  increase  eligibility  by  8  percentage  points  (to  97%)  and  the  average  male  Soldier 
who  would  qualify  for  training  would  have  a  somewhat  lower  chance  of  passing  (71%). 
Raising  the  cutoff  score  by  five  points  (ST  =  97),  the  average  male  Soldier  would  have  a 
somewhat  higher  chance  of  passing  (77%),  but  eligibility  would  be  reduced  by  about  1 1 


Table  2.  91Q:  Probabi 
population)  will  pass  t 

lity  that  a  Regular  Anny  Soldier  (from  the  larger  Army  contract 
le  course  based  on  the  binary  logistic  model 

Cutoff  = 

82 

Cutoff = 

87 

Cutoff = 

92 

Cutoff = 

97 

Cutoff = 
102 

Percent  Eligible 
(Regular  Army) 

99.8% 

97.1% 

89.0% 

78.2% 

64.9%o 

Mean 

106.79 

107.41 

109.06 

111.14 

113.60 

Passing  Rates: 

High  School  Male 

mmm 

sm 

College  Male 

mmm 

K19 

High  School  Female 

■MW 

College  Female 

Mm.1* 

mmm 

percentage  points 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
male  Regular  Army  Soldiers  by  education,  including  upper  and  lower  bounds  based  on 
the  standard  error  of  the  estimated  ST  coefficient.  For  a  particular  ST  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
corresponding  probability  of  passing.  For  example,  a  ST  score  of  100  corresponds  to  a 
passing  probability  of  about  55%  for  a  high  school  educated,  Regular  Army  male  Soldier. 
One  can  see  that  Soldiers  with  a  low  ST  score  stand  very  little  chance  of  passing  the 
course.  As  ST  scores  increase,  the  chance  that  a  Soldier  will  pass  the  course  increases 
sharply.  In  addition,  the  graph  illustrates  that  Soldiers  with  some  college  education  have  a 
noticeably  better  chance  of  passing  this  course. 
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Figure  1:  Predicted  Probalility  of  Passing:  Male  Soldiers 


—  High  School  -  -  -  HS  Upper  -  -  -  HS  Lower  College  —  -  Col  Upper  —  -  Col  Lower 


Figure  2  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
high  school  educated  Soldiers  by  gender,  including  upper  and  lower  bounds  based  upon 
the  standard  error  of  the  estimated  ST  coefficient.  One  can  see  that  Soldiers  with  a  low 
ST  score  stand  very  little  chance  of  passing  the  course.  As  ST  scores  increase,  the  chance 
that  a  Soldier  will  pass  the  course  increases  sharply.  In  addition,  the  graph  illustrates  that 
female  Soldiers  have  a  noticeably  better  chance  of  passing  this  course. 
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Figure  2:  Predicted  Probalility  of  Passing:  High  School  Soldiers 


■'  Male  -  -  -  Male  Upper  -  -  -  Male  Lower  Female  —  ~  Female  Upper  —  ■  Female  Lower 


Figure  3  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
Soldiers  by  education  and  gender.  The  graph  illustrates  that  women  with  some  college 
education  have  the  best  chance  at  passing  the  course,  while  high  school  educated  males, 
the  modal  students,  have  the  lowest  chance  of  passing.  Figure  3  also  illustrates  the 
magnitude  of  the  effect  brought  about  by  gender  and  education.  At  the  current  cutoff 
score  (ST  =  92)  college  educated  female  Soldiers  have  more  than  a  30  percentage  point 
better  chance  of  passing  the  course  compared  to  high  school  educated  males.  Even  at  a 
ST  score  as  high  as  120,  college  educated  women  still  have  approximately  a  10 
percentage  point  better  chance  of  passing  than  high  school  educated  males. 
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Appendix:  Soldier  Characteristics  (91Q) 


Gender 

Male 

Female 

Outcome 

fail 

25.3% 

19.2% 

pass 

74.7% 

80.8% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

pass 

24.6% 

75.4% 

1 1 .5% 

88.5% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

pass 

24.0% 

76.0% 

19.2% 

80.8% 

23.8% 

76.2% 
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91R:  Veterinary  Food  Inspection  Specialist 


The  final  sample  included  507  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (94.5%)  or  failed  for  academic  reasons  (5.5%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  of  the  Soldier  in  MOS  91R  were  male  (61%,  39%  female), 
most  had  a  high  school  diploma  but  not  more  education  (82%,  15%  some  college,  2% 
GED  or  less),  and  most  were  from  Regular  Army  (75%,  25%  Army  Reserve).  The 
governing  AA  composite,  Skilled  Technical  (ST),  for  this  MOS  has  a  cutoff  score  of  96; 
the  sample  mean  is  105.43  (standard  deviation  =  8.234).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 


Table  1.  91R:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

6.524* 

Log  Likelihood 

210.094 

Nagelkerke  R  Square 

.0 

137 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-4.814 

.008 

2.162 

GENDER 

.338 

1.402 

.670 

ST 

.073 

1.075 

5.241* 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 


A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  a  model,  including  the  ST  composite  and  gender  (GENDER  = 
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female),  accounts  for  about  four  percent  of  the  variation  in  the  dependent  variable47.  This 
model  is  statistically  significant  (%2  =  6.524,  P  =  .037),  with  limited  explanatory  power 
(Nagelkerke  R2  =  .037). 

There  is  a  statistically  significant  effect  for  the  ST  composite.  The  model 
suggests  that  having  a  higher  ST  composite  score  increases  the  odds  of  passing.  At  the 
mean  ST  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  8%  in  the 
odds  of  passing  the  course,  and  a  five-point  increase  in  ST  would  increase  the  odds  of 
passing  by  44%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

/  (1  +  Q(P’x)y 

The  following  analysis  is  focused  upon  male  Soldiers  with  a  high  school  diploma  (the 
modal  categories)  from  the  Regular  Army.  Based  on  the  model,  when  the  cutoff  score  is 
at  its  current  level  (ST  =  96),  male  Soldiers  with  a  high  school  diploma  and  an  average 
ST  score  (ST  =  108.67)  have  approximately  a  96%  chance  of  passing.  Currently,  about 
81%  of  Soldiers  are  eligible  for  MOS  91R  assignment  at  the  current  cutoff  (ST  =  96). 
Lowering  the  cutoff  by  five  points  (ST  =  91)  would  increase  eligibility  by  ten  percentage 
points  (to  91%)  and  the  average  male  Soldier  who  would  qualify  for  training  would  have 
about  the  same  chance  of  passing  (96%).  Raising  the  cutoff  score  by  five  points  (ST  = 


47  A  full  model  was  initially  estimated  including  the  ST  composite,  gender,  level  of  education  (i.e.  some 
college)  and  Army  Reserve  membership.  However,  this  model  was  not  statistically  significant  (y2  =  8.079, 
P  =  .089). 
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Table  2.  91R:  Probabi 
population)  will  pass  t 

lity  that  a  Regular  Army  Soldier  (from  the  larger  Anny  contract 
le  course  based  on  the  binary  logistic  model 

Cutoff  = 

86 

Cutoff = 

91 

Cutoff = 

96 

Cutoff = 
101 

Cutoff = 
106 

Percent  Eligible 
(Regular  Army) 

98.1% 

91.0% 

80.5% 

67.8% 

52.7% 

Mean 

106.77 

107.18 

108.67 

110.17 

113.07 

Passing  Rates: 

High  school  male 

95.2% 

95.3% 

95.8% 

96.3% 

96.9% 

101),  the  average  male  Soldier  would  have  about  the  same  chance  of  passing  (96%)  but 
eligibility  would  fall  by  about  13  percentage  points. 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
male  Regular  Army  Soldiers,  including  upper  and  lower  bounds  based  upon  the  standard 
error  of  the  estimated  ST  coefficient.  For  a  particular  ST  score,  trace  a  vertical  line  up  to 
the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the  corresponding 
probability  of  passing.  For  example,  an  ST  score  of  100  corresponds  to  a  passing 
probability  of  about  93%  for  a  high  school  educated,  Regular  Army  male  Soldier.  One 
can  see  that  male  Soldiers  with  a  low  ST  score  still  have  a  moderate  chance  of  passing 
the  course.  As  ST  scores  increase,  the  chance  that  a  Regular  Army  male  Soldier  will  pass 
the  course  increases  somewhat  quickly.  By  the  time  the  ST  score  reaches  the  current 
cutoff  score  (ST  =  96),  Soldiers  already  stand  a  good  chance  of  passing  the  course. 
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Probability  of  passing 


Figure  1:  Predicted  Probability  of  Passing:  Male,  Regular  Army 


ST  composite  score 


I^^^Male  High  School  -  -  -  M/HS  Upper  -  -  -  M/HS  Lower 
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Appendix:  Soldiers  Characteristics  (91R) 


Gender 

Male 

Female 

Outcome 

fail 

5.8% 

5.1% 

pass 

94.2% 

94.9% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

pass 

.0% 

100.0% 

5.7% 

94.3% 

5.1% 

94.9% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

pass 

6.5% 

93.5% 

2.4% 

97.6% 

.0% 

.0% 
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91 S:  Preventative  Medicine  Specialist 


The  final  sample  included  452  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (94.9%)  or  failed  for  academic  reasons  (5.1%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  Soldiers  in  MOS  91S  were  almost  evenly  split  (52%  female,  48% 
male),  most  had  a  high  school  diploma  but  not  more  education  (75%,  23%  some  college, 
2%  GED  or  less),  and  most  were  from  Regular  Army  (64%,  15%  National  Guard,  21% 
Army  Reserve).  The  governing  AA  composite,  Skilled  Technical  (ST),  for  this  MOS  has 
a  cutoff  score  of  102;  the  sample  mean  is  1 1 1.94  (standard  deviation  =  8.255).  See  the 
Appendix  for  a  description  of  Soldier  characteristics  for  this  MOS  training  sample. 


Table  1.  9  IS:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

1 1.572** 

Log  Likelihood 

170.234 

Nagelkerke  R  Square 

.076 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-9.763 

.000 

4.766** 

GENDER 

-0.79 

.924 

.032 

ST 

.117 

1.124 

7.970** 

*  =  p  <  .05  ***=p<.001 

**  =p  <  .01 


A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  a  model,  including  the  ST  composite  and  gender  (GENDER  = 
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48 

female),  accounts  for  almost  eight  percent  of  the  variation  in  the  dependent  variable  . 
This  model  is  statistically  significant  (x  =  1 1.572,  P  =  .003),  and  has  moderate 
explanatory  power  (Nagelkerke  R“  =  .076). 

There  is  a  statistically  significant  effect  for  the  ST  composite.  The  model 
suggests  that  having  a  higher  ST  composite  score  increases  the  odds  of  passing.  At  the 
mean  ST  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  12%  in 
the  odds  of  passing  the  course,  and  a  five-point  increase  in  ST  would  increase  the  odds  of 
passing  by  79%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

/  (1  +  Q(P’x)y 

The  following  analysis  is  focused  upon  male  Soldiers  with  a  high  school  diploma  from 
the  Regular  Army.  Based  on  the  model,  when  the  cutoff  score  is  at  its  current  level  (ST  = 
102),  male  Soldiers  with  a  high  school  diploma  and  an  average  ST  score  (ST  =  1 13.60) 
have  approximately  a  97%  chance  of  passing.  Currently,  about  65%  of  Soldiers  are 
eligible  for  MOS  91S  assignment  at  the  current  cutoff  (ST  =  102).  Lowering  the  cutoff 
by  five  points  (ST  =  97)  would  increase  eligibility  by  13  percentage  points  (to  78%)  and 
the  average  male  Soldier  who  would  qualify  for  training  would  have  about  the  same 
chance  of  passing  (96%).  Raising  the  cutoff  score  by  five  points  (ST  =  107),  the  average 


48  Variables  for  education  status  (GED  or  less,  and  some  college)  and  Army  component  (National  Guard  or 
Army  Reserve  membership)  were  excluded  because  there  were  an  insufficient  number  of  failures  in  each 
variable  category  from  which  to  make  reliable  estimates. 
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Table  2.  9 IS:  Probabi 
population)  will  pass  t 

lity  that  a  Regular  Anny  Soldier  (from  the  larger  Army  contract 
le  course  based  on  the  binary  logistic  model 

Cutoff  = 

92 

Cutoff = 

97 

Cutoff = 
102 

Cutoff = 
107 

Cutoff = 
112 

Percent  Eligible 
( Regular  Army) 

89.0% 

78.2% 

64.9% 

49.6% 

34.7% 

Mean 

109.06 

111.14 

113.60 

116.55 

119.83 

Passing  Rates: 

High  School  Male 

95.1% 

96.1% 

97.1% 

97.9% 

98.6% 

male  Soldier  would  have  about  the  same  chance  of  passing  (98%)  but  eligibility  would 
fall  by  about  15  percentage  points. 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
male  Regular  Army  Soldiers,  including  upper  and  lower  bounds  based  upon  the  standard 
error  of  the  estimated  ST  coefficient.  For  a  particular  ST  score,  trace  a  vertical  line  up  to 
the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the  corresponding 
probability  of  passing.  For  example,  an  ST  score  of  100  corresponds  to  a  passing 
probability  of  about  87%  for  a  high  school  educated,  Regular  Army  male  Soldier.  One 
can  see  that  male  Soldiers  with  a  low  ST  score  have  a  low  chance  of  passing  the  course. 
As  ST  scores  increase,  the  chance  that  a  Regular  Anny  male  Soldier  will  pass  the  course 
increases  somewhat  quickly.  By  the  time  the  ST  score  reaches  the  current  cutoff  score 
(ST  =  96),  male  Soldiers  already  stand  a  good  chance  of  passing  the  course. 
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Figure  1:  Predicted  Probability  of  Passing:  Male  Regular  Army 


ST  composite  score 


-  -  -  Male  Upper  -  -  -  Male  Lower 
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Appendix:  Soldier  Characteristics  (9 IS) 


Gender 

Male 

Female 

Outcome 

fail 

4.2% 

5.9% 

pass 

95.8% 

94.1% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

pass 

9.1% 

90.9% 

6.2% 

93.8% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

pass 

6.2% 

93.8% 

2.1% 

97.9% 

4.5% 

95.5% 
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91T:  Animal  Care  Specialist 


The  final  sample  included  376  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (86.2%)  or  failed  for  academic  reasons  (13.8%)  were 
included  in  the  analysis  sample  of  376  Soldiers  because  we  assume  that  non-completion 
for  non-academic  reasons  is  not  related  to  cognitive  criteria. 

Most  of  the  Soldiers  in  MOS  91T  were  female  (65%,  35%  male),  most  had  a  high 
school  diploma  but  not  more  education  (81%,  19%  some  college,  <1%  GED  or  less),  and 
most  were  from  Regular  Army  (86%,  14%  Army  Reserve).  The  governing  AA 
composite,  Skilled  Technical  (ST),  for  this  MOS  has  a  cutoff  score  of  92;  the  sample 
mean  is  1 1 1 .95  (standard  deviation  =  9.279).  See  Appendix  for  a  description  of  Soldier 
characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 


Table  1.  9  IT:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

47.263*** 

Log  Likelihood 

254.937 

Nagelkerke  R  Square 

.214 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-13.097 

.000 

28.034*** 

GENDER 

.690 

1.994 

4.176* 

ST 

.134 

1.143 

34.194*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 


Table  1  and  indicate  that  a  model,  including  the  ST  composite  and  gender  (GENDER  = 
female),  accounts  for  about  2 1  percent  of  the  variation  in  the  dependent  variable.  This 
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model  is  statistically  significant  (%  =  47.263,  P  <  .001),  and  it  has  moderate  explanatory 
power  (Nagelkerke  R  =  .214). 

There  is  a  statistically  significant  effect  for  the  ST  composite.  The  model 
suggests  that  having  a  higher  ST  composite  score  increases  the  odds  of  passing.  At  the 
mean  ST  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  14%  in 
the  odds  of  passing  the  course,  and  a  five-point  increase  in  ST  would  increase  the  odds  of 
passing  by  95%.  The  model  also  suggests  that  females  have  increased  odds  of  passing 
the  course  (100%  greater  than  males). 

Policy  Analysis.  Table  2  and  Figure  1  report  the  probability  that  the  average 
Soldier  from  the  larger  successful  applicant  population  would  pass  the  course  based  upon 
the  binary  logistic  model.  These  were  calculated  using  the  formula  for  finding  probability 
in  a  binary  logistic  model,  which  is 

/  (1  +  Q(P’x)y 

The  following  analysis  is  focused  upon  Soldiers  with  a  high  school  diploma  from  the 
Regular  Army.  Based  on  the  model,  when  the  cutoff  score  is  at  its  current  level  (ST  = 
92),  male  Soldiers  with  a  high  school  diploma  and  an  average  ST  score  (ST  =  109.06) 
have  approximately  an  82%  chance  of  passing;  female  soldiers  have  a  95%  chance  of 
passing.  Currently,  about  89%  of  Soldiers  are  eligible  for  MOS  9  IT  assignment  at  the 
current  cutoff  (ST  =  92).  Lowering  the  cutoff  by  five  points  (ST  =  87)  would  increase 
eligibility  by  8  percentage  points  (to  97%)  while  the  average  male  Soldier  who  would 
qualify  for  training  would  have  a  somewhat  lower  chance  of  passing  (79%);  the  average 
female  Soldier  would  have  the  same  passing  rate  (94%)  as  at  the  current  cutoff.  Raising 
the  cutoff  score  by  five  points  (ST  =  97),  the  average  qualifying  male  and  female  Soldiers 
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would  have  slightly  higher  chances  of  passing  but  eligibility  would  fall  by  1 1  percentage 
points. 


Table  2.  9 IT:  Probability  that  a  Soldier  (from  the  larger  Army  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model 

Cutoff  = 

82 

Cutoff = 

87 

Cutoff = 

92 

Cutoff = 

97 

Cutoff = 
102 

Percent  Eligible 
(Regular  Army) 

99.8% 

97.1% 

89.0% 

78.2% 

64.9%o 

Mean 

106.79 

107.41 

109.06 

111.14 

113.60% 

Passing  Rates: 

Male 

77.1% 

78.5% 

82.0% 

85.8% 

89.3% 

Female 

93.2% 

93.7% 

94.9% 

96.1% 

97.2% 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
male  and  female  Regular  Army  Soldiers,  including  upper  and  lower  bounds  based  upon 
the  standard  error  of  the  estimated  ST  coefficient.  For  a  particular  ST  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
corresponding  probability  of  passing.  For  example,  a  ST  score  of  100  corresponds  to  a 
passing  probability  of  about  85%  for  a  high  school  educated,  Regular  Army  female 
Soldier.  One  can  see  that  male  Soldiers  with  a  low  ST  score  have  a  low  chance  of  passing 
the  course.  As  ST  scores  increase,  the  chance  that  a  Regular  Army  Soldier  will  pass  the 
course  increases  somewhat  quickly.  Figure  1,  also  demonstrates  that  the  chance  of  a 
female  Soldier  passing  increase  more  quickly  than  that  of  a  male  Soldier  leading  to  a 
large  difference  at  the  current  cutoff  score  (ST  =  92),  where  female  Soldiers  have  over  a 
30%  better  chance  of  passing  the  course  than  males. 
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Chance  of  Passing 


Figure  1 :  Predicted  Probability  of  Passing:  Regular  Army  Soldiers  with  a  High  School  Diploma 


ST  Composite  Scores 


Male 

-  -  -  Male  Upper 

-  -  -  Male  Lower 

^^^“Female 

—  -  Female  Upper 

—  -  Female  Upper 
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Appendix:  Soldier  Characteristics  (9 IT) 


Gender 

Male 

Female 

Outcome 

fail 

16.2% 

12.6% 

pass 

83.8% 

87.4% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

pass 

.0% 

100.0% 

16.4% 

83.6% 

2.8% 

97.2% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

pass 

15.8% 

84.2% 

1 .9% 

98.1% 

sO 

O'  o' 

O  O 
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91 W:  Health  Care  Specialist 


The  final  sample  included  10,972  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  for  this  MOS  during  a  period  from  2001 
to  2004.  Only  Soldiers  who  graduated  (85.7%)  or  failed  for  academic  reasons  (14.3%) 
were  included  in  the  analysis  sample  of  10,972  Soldiers  because  we  assume  that  non¬ 
completion  for  non-academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (74%,  26%  female),  most  had  a  high  school 
diploma  but  not  more  education  (80%,  17%  some  college,  3%  GED  or  less),  and  most 
were  from  were  from  the  Regular  Anny  (65%,  19%  National  Guard,  16%  Reserve).  The 
governing  AA  composites,  Skilled  Technical  (ST)  and  General  Technical  (GT),  for  this 
MOS  have  cutoff  scores  of  102  and  110  respectively;  the  sample  mean  is  1 15.46 
(standard  deviation  =  8.730)  for  ST  and  1 16.43  (standard  deviation  =  7.989)  for  GT.  The 
ST  and  GT  composites  are  correlated  at  the  .826  level  (p  <  .001).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 

Model  Estimation.  A  binary  logistic  regression  model  was  estimated  to  explain 
pass/fail  AIT  outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results 
are  reported  in  Table  1  and  indicate  that  the  stepwise  model49,  including  the  ST 
composite,  education  level  (COLLEGE  =  some  college),  Army  component  (GUARD  = 
National  Guard;  RESERVE  =  Army  Reserve),  and  gender  (GENDER  =  female), 
accounts  for  about  14  percent  of  the  variation  in  the  dependent  variable.  This  model  is 
statistically  significant 


49  Initially  we  estimated  a  model  including  ST,  GT,  GED,  COLLEGE,  GUARD,  RESERVE,  and 
GENDER.  This  is  reported  in  the  Appendix  (Table  la).  In  that  model  the  coefficient  for  GT  is  negative, 
very  small,  and  non-significant.  Given  the  correlation  between  ST  and  GT,  such  an  effect  can  be  indicative 
of  collinearity.  The  stepwise  model  retained  ST  while  excluding  GT,  and  thus  confirmed  this  presumption. 
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Table  1.  91W:  Results  of  the  forward  stepwise 
binary  logistic  prediction  model 

Chi-  Square 

876.763*** 

Log  Likelihood 

8114.123 

Nagelkerke  R  Square 

.137 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-9.613 

.000 

385.898*** 

COLLEGE 

.613 

1.846 

39.146*** 

RESERVE 

.531 

1.700 

38.879*** 

GUARD 

.856 

2.354 

83.340*** 

GENDER 

.481 

1.618 

48.914*** 

ST 

.097 

1.102 

500.069*** 

*  =  p  <  .05  ***=p<.001 

**  =p  <  .01 

Of  =876.763,  P  <  .001),  and  has  moderate  explanatory  power  (Nagelkerl 


keR2  =  .137). 


There  is  a  statistically  significant  effect  for  the  ST  composite.50  The  model 
suggests  that  having  a  higher  ST  composite  score  increases  the  odds  of  passing.  At  the 
mean  ST  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  10%  in 
the  odds  of  passing  the  course,  and  a  five-point  increase  in  ST  would  increase  the  odds  of 
passing  by  62%.  There  are  also  statistically  significant  effects  for  COLLEGE,  GUARD, 
RESERVE,  and  GENDER,  with  increased  odds  of  passing  the  course  at  85%,  135%, 

70%,  and  62%  (for  females),  respectively. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 


5(1  There  is  also  a  statistically  significant  effect  for  the  GT  composite  when  inserted  in  place  of  the  ST 
composite.  See  the  Appendix  (Table  lb). 
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Q(P’x)  /  (1  +  Q(P’x)y 


The  following  analysis  is  focused  upon  male  and  female  Soldiers  with  a  high  school 
diploma  or  some  college  education  from  the  Regular  Army.  Based  on  the  model,  when 
the  cutoff  score  is  at  its  current  level  (ST  =  102,  GT  =  110),  male  Soldiers  with  a  high 
school  diploma  (the  modal  categories)  and  an  average  ST  score  (ST  =  1 18.12)  have 
approximately  an  86%  chance  of  passing.  Currently,  about  33%  of  Soldiers  are  eligible 
for  MOS  91W  assignment  at  the  current  cutoff  (ST  =  102,  GT  =  1 10).  Lowering  the 
cutoff  by  five  points  (ST  =  97,  GT  =  105)  would  increase  eligibility  by  14  percentage 
points  (to  47%)  and  the  average  male  Soldier  who  would  qualify  for  training  would  have 
a  slightly  lower  chance  of  passing  (83%).  Raising  the  ST  cutoff  score  by  five  points  and 
keeping  GT  at  its  current  level  (ST  =  107,  GT  =110)  leads  to  only  small  increases  in 
passing  rates,  and  would  reduce  eligibility  by  almost  five  percentage  points. 


Table  2.  91W:  Probability  that  a  Regular  Army  Soldier  (from  the  larger  successful 
applicant  population)  will  pass  the  course  based  on  the  binary  logistic  model 

Cutoff: 

ST  =  92 
GT=  100 

Cutoff: 

ST  =  97 
GT=  105 

Cutoff 

ST  =  102 
GT=  110 

Cutoff 

ST  =  107 
GT=  110 

Cutoff 

ST  =  112 
GT=  110 

Percent  Eligible 
(Regular  Army) 

65.1% 

47.2% 

32.7% 

28.2%o 

22.7% 

Mean 

112.43 

115.24 

118.12 

119.46 

121.32 

Passing  Rates: 

High  School  Male 

78.5% 

82.7% 

86.3% 

87.8% 

89.6% 

College  Male 

87.1% 

89.8% 

92.1% 

93.0% 

94.1% 

High  School  Female 

85.5% 

88.6% 

91.1% 

92.1% 

93.3% 

College  Female 

91.6% 

93.5% 

95.0% 

95.6% 

96.3% 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
male  Regular  Army  Soldiers  by  education,  including  upper  and  lower  bounds  based  upon 
the  standard  error  of  the  estimated  ST  coefficient.  For  a  particular  ST  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
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corresponding  probability  of  passing.  For  example,  an  ST  score  of  100  corresponds  to  a 
passing  probability  of  about  50%  for  a  high  school  educated.  Regular  Army  male  Soldier. 
One  can  see  that  Soldiers  with  a  low  ST  score  stand  very  little  chance  of  passing  the 
course.  As  ST  scores  increase,  the  chance  that  a  Soldier  will  pass  the  course  increases 
sharply.  In  addition,  the  graph  illustrates  that  Soldiers  with  some  college  education  have  a 
noticeably  better  chance  of  passing  this  course. 


Figure  1:  Predicted  Probability  of  Passing:  Male  Soldiers 


—  High  School  -  -  -  HS  Upper  -  -  -  HS  Lower  College  —  -  Col  Upper  —  -  Col  Lower 


Figure  2  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
high  school  educated  Soldiers  by  gender,  including  upper  and  lower  bounds  based  upon 
the  standard  error  of  the  estimated  ST  coefficient.  The  graph  illustrates  that  female 
Soldiers  have  a  noticeably  better  chance  of  passing  this  course. 
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Figure  2:  Predicted  Probability  of  Passing:  High  School  Soldiers 


■'  Male  -  -  -  Male  Upper  -  -  ■  Male  Lower  ■—  Female  —  -  Female  Upper  ■  Female  Lower 


Figure  3  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
Soldiers  by  education  and  gender.  The  graph  illustrates  that  women  with  some  college 
education  have  the  best  chance  at  passing  the  course,  while  high  school  educated  males, 
the  modal  students,  have  the  lowest  chance  of  passing.  Figure  3  also  illustrates  the 
magnitude  of  the  effect  brought  about  by  gender  and  education.  At  the  current  cutoff 
score  (ST  =  102)  college  educated  female  Soldiers  have  more  than  a  30  percentage  point 
better  chance  of  passing  the  course  compared  to  high  school  educated  males.  Even  at  an 
ST  score  as  high  as  120,  college  educated  women  still  have  approximately  a  25 
percentage  point  better  chance  of  passing  than  high  school  educated  males. 
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Testing  a  Central  Assumption  of  the  Approach 


Central  to  our  approach  has  been  the  estimation  of  model  parameters  using  only 
academic  passes  and  failure  cases  -  we  specifically  exclude  failure  cases  that  have  been 
identified  with  non-academic  reasons  for  failure.  In  doing  so  we  assume  that  non¬ 
completion  for  non-academic  reasons  is  not  related  to  cognitive  criteria.  In  this  section 
we  report  on  efforts  to  examine  the  reasonableness  of  this  assumption. 

When  we  include  non-academic  failure  cases  in  the  data  sample,  we  would  expect 
the  goodness-of-fit  to  deteriorate  and  to  see  smaller  predicted  composite  effects,  if  higher 
scoring  Soldiers  predominate  in  these  “additional”  cases.  Alternatively,  if  lower  scoring 
Soldiers  predominate  in  these  cases,  it  is  possible  to  see  little  or  no  deterioration  in  the  fit 
of  the  model.  In  Table  3  we  present  the  regression  model  results  when  an  additional 
2,962  non-academic  failure  cases  are  included  in  the  analysis  sample  (for  a  total  of 
13,934  cases).  There  is  a  clear  deterioration  in  the  explanatory  power  of  the  model:  the 


Table  3.  91W:  Results  of  the  forward  stepwise 
binary  logistic  prediction  model  including  both 
academic  and  non-academic  failure  cases 


Chi-  Square 

605.195*** 

Log  Likelihood 

16965.570 

Nagelkerke  R  Square 

.059 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-3.773 

.023 

136.642*** 

GED 

-.610 

.543 

40.320*** 

COLLEGE 

.266 

1.305 

24.801*** 

RESERVE 

.350 

1.420 

44.422*** 

GUARD 

.681 

1.975 

152.199*** 

ST 

.048 

1.050 

153.759*** 

GT 

-.011 

.989 

5.990* 

*  =  p  <  .05  **  =  p  <  .01  ***  =  p  <  .001 
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2 

Nagelkerke  R“  declines  from  0.137  in  the  original  sample  to  .059  in  the  augmented 
sample.  In  addition,  the  ST  effect  in  this  data  sample  is  considerably  weaker  than  in  the 
original  sample  (1.050  versus  1.102). 

An  even  more  extreme  comparison  might  be  between  the  original  sample  and  a 
sample  utilizing  only  the  non-academic  failure  cases  -  i.e.,  to  replace  the  academic  failure 
cases  in  the  original  sample  with  non-academic  failure  cases.  The  hypothesis  is  that  the 
latter  bring  increased  randomness  to  the  data  set,  and  that  the  resulting  regression  model 
will  not  fit  as  well,  and  the  estimated  governing  composite  effects  will  be  weaker.  In 
Table  4  we  present  the  regression  model  results  when  only  non-academic  failure  cases 
are  included  (along  with  academic  pass  cases).  This  data  sample  has  a  total  of  12,369 
observations.  We  find  even  greater  deterioration  in  the  explanatory  power  of  the  model: 
the  Nagelkerke  R"  in  the  “non-academic  failure”  sample  model  is  only  about  one-fifth 
that  of  the  original  sample  model  (.026  versus  .137).  In  addition,  the  ST  composite  effect 
is  considerably  weaker  -  1.029  for  the  sample  including  non-academic  failures  compared 
to  1.102  for  the  original  sample. 
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Table  4.  91W:  Results  of  the  forward  stepwise  binary 
logistic  prediction  model  including  non-academic 
failures  only _ _ 


Chi-  Square 

220.239*** 

Log  Likelihood 

13397.234 

Nagelkerke  R  Square 

.026 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-.341 

.711 

.983 

GED 

-.785 

.456 

58.010*** 

COLLEGE 

.155 

1.168 

6.897** 

GUARD 

.585 

1.795 

86.679*** 

RESERVE 

.262 

1.300 

19.324*** 

GENDER 

-.117 

.890 

5.552* 

ST 

.028 

1.029 

35.915*** 

GT 

-.016 

.984 

9.906** 

*  =  p  <  .05  **  =  p  <  .01  ***=p<.001 
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Appendix:  Additional  Regression  Models 


Table  la.  91W:  Results  of 
prediction  model  includin, 

'the  binary  logistic 
g  all  variables 

Chi-  Square 

876.979*** 

Log  Likelihood 

8113.906 

Nagelkerke  R  Square 

.137 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-9.509 

.000 

309.804*** 

GED 

-0.35 

.965 

.040 

COLLEGE 

.614 

1.847 

39.114*** 

GUARD 

.858 

2.359 

81.683*** 

RESERVE 

.532 

1.703 

37.263*** 

GENDER 

.484 

1.623 

48.938*** 

ST 

.099 

1.104 

249.175*** 

GT 

-.003 

.997 

.190 

*  =  p  <  .05  ***  =  p  <  .00] 

**  =p  <  .01 

[ 

Table  lb.  91W:  Results  of  the  binary  logistic 
prediction  model  excluding  ST 

Chi-  Square 

605.028*** 

Log  Likelihood 

8385.857 

Nagelkerke  R  Square 

.096 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-6.727 

.001 

183.587*** 

GED 

.074 

1.077 

.178 

COLLEGE 

.738 

2.092 

57.405*** 

GUARD 

.966 

2.626 

104.868*** 

RESERVE 

.586 

1.797 

45.871*** 

GENDER 

.191 

1.211 

8.295** 

GT 

.071 

1.073 

267.945*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 

[ 
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Appendix:  Soldier  Characteristics  (91W) 


Gender 

Male 

Female 

Outcome 

fail 

14.6% 

13.2% 

pass 

85.4% 

86.8% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

pass 

13.3% 

86.7% 

15.8% 

84.2% 

7.2% 

92.8% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

pass 

17.2% 

82.8% 

10.6% 

89.4% 

7.1% 

92.9% 
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91 W  Risk  Analysis  of  Policy  Alternatives 


In  this  section  we  extend  the  policy  analysis  with  a  risk  analysis  of  the  policy 
alternatives  considered  using  simulation  methods.  We  employ  the  “@RISK”  software 
package  which  is  designed  as  an  add-in  for  Microsoft  Excel.51  There  are  four  steps  in  the 
risk  analysis. 

The  first  step  is  developing  a  model.  A  regression  model  for  predicting  the 
passing  rate  has  been  specified  and  estimated,  and  the  results  stored  in  Excel  worksheet 
format.  See  Table  1. 

The  second  step  is  identifying  the  uncertainty  inherent  in  both  the  input  variables 
(composite  scores,  Soldier  characteristics)  and  the  corresponding  estimated  parameters 
and  specifying  their  possible  values  with  probability  distributions. 

To  approximate  the  governing  composite  distribution,  we  use  the  larger  Army 
contract  population  (EAF  file:  those  who  contracted  from  Jan  1992  to  Aug  2003).  This 
distribution  can  be  assumed  to  follow  a  normal  distribution.  The  mean  and  standard 
deviation  estimated  from  the  contract  population  and  used  as  input  to  the  simulations  are 
shown  in  Table  R-l  (before  truncation). 


Table  R-l.  Input  infonnation  for  the  simulations 


Mean 

Std  error  / 
deviation 

Constant 

-9.609 

.490 

ST  var 

1 17.13  (before  truncation,  but 
conditioned  on  GT  >=110 

9.133 

ST  coeff 

.097 

.004 

GENDER  var 

26.0%  (female) 

GENDER  coeff 

.481 

.069 

GED  var 

3.3% 

GED  coeff 

-.028 

.174 

COLLEGE  var 

17.1% 

COLLEGE  coeff 

.612 

.098 

RESERVE  var 

16.6% 

RESERVE  coeff 

.534 

.087 

GUARD  var 

18.7% 

GUARD  coeff 

.858 

.095 

51  Software  is  available  from  Palisade  Corporation.  Ncwt'icld,  NY. 
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To  approximate  the  distributions  for  the  demographic  variables  (GENDER,  GED, 
COLLEGE,  RESERVE,  and  GUARD)  we  rely  on  the  training  data  sample  used  to 
estimate  the  regression  model.  These  distributions  can  be  assumed  to  follow  binomial 
distributions  with  means  estimated  from  the  training  data  sample  as  shown  in  Table  R-l. 

To  approximate  the  distributions  for  the  estimated  parameters  in  the  logit  model, 
we  note  that,  under  very  general  conditions,  the  estimates  will  be  asymptotically  normally 
distributed.52  The  estimated  coefficients  (means  and  standard  errors)  are  shown  in  Table 
R-l. 


The  third  step  is  analyzing  the  model  with  simulation  to  determine  the  range  and 
probabilities  of  all  possible  training  outcome  results.  The  results  reported  represent  the 
simulation  of  the  regression  prediction  equation  for  1000  iterations  using  a  fixed  seed.  ^ 
The  comparisons  are  between  the  baseline  case  (ST  =  102)  and  the  policy  cases  (ST  =  97 
and  ST  =  107).  These  cases  are  delineated  by  the  cutoff  score  level  which  serves  as  the 
lower  truncation  point  in  the  governing  composite  input  distribution.  The  simulation 
results  in  Table  R-2  describe  the  effect  on  the  mean  probability  of  passing  as  the  ST 
cutoff  score  level  is  varied,  and  represent  an  improvement  of  the  policy  analysis 
presented  in  Table  2  because  they  reflect  the  simultaneous  effects  of  all  known  sources  of 
uncertainty. 


Table  R-2.  91 W  simulation  results 


ST  =  97 

GT  =  105 

ST  =  102 
GT=  110 

ST  =  107 
GT=  110 

Mean  governing 
composite 

115.38 

118.10 

119.40 

Mean  probability 
of  passing 

83.0% 

86.2% 

87.8% 

Std  deviation 

13.4% 

11.9% 

10.2% 

The  frequency  distributions  of  simulated  effects  are  shown  below  in  the  histogram 
graphs.  The  histogram  portrays  the  full  range  of  outcomes  that  average  to  the  mean 
effect  as  reported.  Looking  at  the  histogram  for  the  baseline  case  (ST  =  102),  probability 
of  passing  is  plotted  on  the  horizontal  axis  and  the  height  of  each  bar  indicates  the 
relative  number  of  Soldiers  at  that  passing  probability.  The  information  in  the  histograms 
is  effectively  summarized  in  Table  R-3.  In  that  table  we  see  the  percentage  of  Soldiers 


52  The  intuition  is  that  logit  estimates  are  derived  from  maximum  likelihood  (ML)  estimation,  and  under 
fairly  general  conditions  ML  estimates  are  consistent  and  asymptotically  normally  distributed. 

53  Latin  Hypercube  (LH)  sampling  is  utilized  in  drawing  samples  for  the  input  distributions.  It  is  designed 
to  accurately  recreate  the  input  distribution  through  sampling  in  fewer  iterations  when  compared  with  the 
Monte  Carlo  method.  The  key  to  LH  sampling  is  stratification  of  the  input  probability  distributions. 


334 


that  meet  or  exceed  a  given  chance  of  passing.  For  example,  looking  at  the  baseline  case 
(ST  =  102),  about  88%  of  Soldiers  have  a  70%  or  greater  chance  of  passing. 


Table  R-3. 

Percent  of  simulated  Soldiers  that  meet  or  exceed 
given  chance  of  passing 


Chance  of 
passing 

ST  =  97 

CN 

O 

II 

1 — 
CO 

ST  =  107 

95.0% 

19.3% 

26.9% 

27.1% 

90.0% 

39.6% 

46.5% 

52.7% 

85.0% 

55.1% 

62.1% 

69.2% 

80.0% 

67.4% 

74.6% 

80.9% 

75.0% 

77.0% 

81.9% 

88.8% 

70.0% 

82.9% 

87.5% 

93.7% 

65.0% 

87.8% 

92.0% 

96.4% 

60.0% 

91.9% 

94.2% 

97.5% 

55.0% 

94.2% 

96.3% 

98.7% 

50.0% 

96.4% 

98.1% 

99.5% 

We  find  that  the  mean  probability  of  passing  in  the  simulation  is  86%  (Table  R-2, 
baseline).  Looking  at  the  histograms,  we  see  the  largest  portion  of  Soldiers  in  the  highest 
range  of  passing  scores,  and  the  remainder  distributed  in  a  stair  step  into  the  middle 
range.  We  note  that  the  mean  simulated  passing  probability  turns  out  to  be  about  the 
same  as  the  static  prediction  at  the  mean  ST  score  (Table  2).  In  the  simulated  predictions 
we  are  calculating  an  average  over  all  Soldiers  and  taking  into  account  all  sources  of 
uncertainty,  whereas  in  the  static  prediction  we  are  calculating  the  probability  of  passing 
only  at  the  mean  ST  score,  ignoring  the  distribution  of  Soldier  scores  etc.  as  well  as 
uncertainty.  While  recognizing  this  variability  is  important,  we  also  note  that  the  policy 
case  impacts  relative  to  the  baseline  are  the  same  in  both  simulation  and  static  prediction 
modes. 


The  fourth  step  is  making  a  decision  based  on  the  results  provided  and  the 
preferences  of  the  school  proponent.  This  involves  the  evaluation  by  the  school 
proponent  of  the  tradeoff  between  increased  eligibility  and  reduced  passing  probabilities 
(and  vice-versa). 
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Distribution  for  PASS  EQUATION  102 


X  <=0.61  X  <=0.98 


Distribution  for  PASS  EQUATION  107 


X  <=0.68  X  <=0.99 
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Distribution  for  PASS  EQUATION  97 


X  <=0.54  X  <=0.98 
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9 IX:  Mental  Health  Specialist 


The  final  sample  included  678  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  for  this  MOS  during  a  period  from  2001 
to  2004.  Only  Soldiers  who  graduated  (84.3%)  or  failed  for  academic  reasons  (15.7%) 
were  included  in  the  analysis  sample  of  678  Soldiers  because  we  assume  that  non¬ 
completion  for  non-academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (55%,  45%  female),  most  had  a  high  school 
diploma  but  not  more  education  (69%,  29%  some  college,  2%  GED  or  less),  and  most 
were  from  were  from  the  Regular  Anny  (55%,  7%  National  Guard,  38%  Reserve).  The 
governing  AA  composite,  Skilled  Technical  (ST)),  for  this  MOS  has  a  cutoff  score  of 
102;  the  sample  mean  is  1 12.90  (standard  deviation  =  10.227).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 


Table  1.  9 IX:  Results  of  the  forward  stepwise 
binary  logistic  prediction  model _ 


Chi-  Square 

66.845*** 

Log  Likelihood 

524.415 

Nagelkerke  R  Square 

.161 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-5.920 

.003 

11.652** 

GED 

-.179 

.836 

.065 

COLLEGE 

1.331 

3.786 

16.203*** 

RESERVE 

.554 

1.740 

5.233* 

GUARD 

.891 

2.437 

2.933 

GENDER 

1.294 

3.647 

26.314*** 

ST 

.059 

1.061 

14.725** 

*  =  p  <  .05  ***  =  p<.001 

**  =  p  <  .01 
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Table  1  and  indicate  that  the  model,  including  the  ST  composite,  education  level  (GED  = 
GED  or  less,  COLLEGE  =  some  college),  Army  component  (GUARD  =  National  Guard; 
RESERVE  =  Army  Reserve)  and  gender  (GENDER  =  female),  accounts  for  about  16 
percent  of  the  variation  in  the  dependent  variable.  This  model  is  statistically  significant 
(X2  =  66.845,  P  <  .001),  and  has  moderate  explanatory  power  (Nagelkerke  R2  =  .161). 

There  is  a  statistically  significant  effect  for  the  ST  composite.  The  model 
suggests  that  having  a  higher  ST  composite  score  increases  the  odds  of  passing.  At  the 
mean  ST  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  6%  in  the 
odds  of  passing  the  course,  and  a  five-point  increase  in  ST  would  increase  the  odds  of 
passing  by  34%.  There  are  also  statistically  significant  effects  for  COLLEGE, 
RESERVE,  and  GENDER,  with  increased  odds  of  passing  the  course  at  279%,  74%,  and 
265%  (for  females),  respectively. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

q(P’x)  /  (1  +  e(p’x}). 

The  following  analysis  is  focused  upon  male  and  female  Soldiers  with  a  high  school 
diploma  or  some  college  education  from  the  Regular  Army.  When  the  cutoff  score  is  at 
its  current  level  (ST  =  102),  male  (female)  Soldiers  with  a  high  school  diploma  (the 
modal  categories)  and  an  average  ST  score  (ST  =  1 13.60)  have  approximately  a  69% 
(89%)  chance  of  passing.  Currently,  about  65%  of  Soldiers  are  eligible  for  MOS  9 IX 
assignment  at  the  current  cutoff  (ST  =  102).  Lowering  the  cutoff  by  five  points  (ST  =  97) 
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would  increase  eligibility  by  13  percentage  points  (to  78%)  and  the  average  male 
(female)  Soldier  who  would  qualify  for  training  would  have  a  somewhat  lower  chance  of 
passing  (65%)  (87%).  Raising  the  cutoff  score  by  five  points  (ST  =  107),  the  average 
male  (female)  Soldier  would  have  a  somewhat  higher  chance  of  passing  (72%)  (91%), 
but  eligibility  would  fall  by  about  15  percentage  points. 


Table  2.  9 IX:  Probabi 
population)  will  pass  t 

lity  that  a  Regular  Anny  Soldier  (from  the  larger  Army  contract 
le  course  based  on  the  binary  logistic  model 

Cutoff: 

ST  =  92 

Cutoff: 

ST  =  97 

Cutoff 

ST  =  102 

Cutoff 

ST  =  107 

Cutoff 

ST  =  112 

Percent  Eligible 
(Regular  Army) 

89.0% 

78.2% 

64.5% 

49.6% 

34.7% 

Mean 

109.06 

111.14 

113.60 

116.55 

119.83 

Passing  Rates: 

High  School  Male 

mmm, 

mmm 

College  Male 

mmm 

High  School  Female 

mmm 

■M— 

College  Female 

mmm. 

mmm 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
male  Regular  Army  Soldiers  by  education,  including  upper  and  lower  bounds  based  upon 
the  standard  error  of  the  estimated  ST  coefficient.  Far  a  particular  ST  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
corresponding  probability  of  passing.  For  example,  an  ST  score  of  100  corresponds  to  a 
passing  probability  of  about  50%  for  a  high  school  educated,  Regular  Army  male  Soldier. 
One  can  see  that  Soldiers  with  a  low  ST  score  stand  very  little  chance  of  passing  the 
course.  As  ST  scores  increase,  the  chance  that  a  Soldier  will  pass  the  course  increases 
gradually.  In  addition,  the  graph  illustrates  that  Soldiers  with  some  college  education 
have  a  noticeably  better  chance  of  passing  this  course. 


340 


Chance  of  Passing 


Figure  1:  Predicted  Probability  of  Passing:  Male  Soldiers 


—  High  School  -  -  -  HS  Upper  -  -  -  HS  Lower  College  —  -  Col  Upper  —  -  Col  Lower 


Figure  2  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
high  school  educated  Soldiers  by  gender,  including  upper  and  lower  bounds.  The  graph 
illustrates  that  female  Soldiers  have  a  noticeably  better  chance  of  passing  this  course. 
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Figure  2:  Predicted  Probability  of  Passing:  High  School  Soldiers 


ST  Composite 


•Male  -  -  -  Male  Upper  -  -  -  Male  Lower  Female  — —  -  Female  Upper  —  -  Female  Lower 


Figure  3  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
Soldiers  by  education  and  gender.  The  graph  illustrates  that  women  with  some  college 
education  have  the  best  chance  at  passing  the  course,  while  high  school  educated  males, 
the  modal  students,  have  the  lowest  chance  of  passing.  College  educated  male  Soldiers 
and  high  school  educated  female  Soldiers  have  essentially  the  same  scores.  Figure  3  also 
illustrates  the  magnitude  of  the  effect  brought  about  by  gender  and  education.  At  the 
current  cutoff  score  (ST  =  102)  college  educated  female  Soldiers  have  more  than  a  30 
percentage  point  better  chance  of  passing  the  course  compared  to  high  school  educated 
males.  Even  at  a  ST  score  as  high  as  120,  college  educated  women  still  have 
approximately  a  20  percentage  point  better  chance  of  passing  than  high  school  educated 
males. 
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Figure  3:  HS  vs.  College  &  Male  vs.  Female 


ST  composite  score 


■3 


Appendix:  Soldier  Characteristics  (9 IX) 


Gender 

Male 

Female 

fail 

pass 

8.9% 

91.1% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

pass 

23.1% 

76.9% 

19.7% 

80.3% 

6.1% 

93.9% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

pass 

17.5% 

82.5% 

14.0% 

86.0% 

1 1 .4% 

88.6% 
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88M:  Motor  Transport  Operator 


The  final  sample  included  14310  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (98.6%)  or  failed  for  academic  reasons  (1.4%)  were 
included  in  the  analysis  sample  of  14310  Soldiers  because  we  assume  that  non¬ 
completion  for  non-academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  of  the  Soldiers  were  male  (73%,  27%  female),  most  had  a 
high  school  diploma  but  not  more  education  (90%,  8%  some  college,  2%  GED  or  less), 
and  the  greatest  number  were  from  National  Guard  (40%,  34%  Regular  Army,  26% 

Army  Reserve).  The  governing  AA  composite,  Operator/Food  (OF),  for  this  MOS  has  a 
cutoff  score  of  85;  the  sample  mean  is  101.18  (standard  deviation  =  12.637).  See  the 
Appendix  for  a  description  of  Soldier  characteristics  for  this  MOS  training  sample. 


Table  1.  88M:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

82.021*** 

Log  Likelihood 

2023.329 

Nagelkerke  R  Square 

.0 

142 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

.484 

1.622 

.765 

GED 

.933 

2.542 

1.532 

COLLEGE 

.286 

1.330 

.757 

GUARD 

.893 

2.443 

24.104*** 

RESERVE 

.441 

1.554 

5.850* 

GENDER 

-.019 

.981 

.015 

OF 

.034 

1.035 

36.060*** 

*  =  p  <  .05  ***  =  p<.001 

**  =  p  <  .01 


A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
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Table  1  and  indicate  that  a  model,  including  the  OF  composite,  as  well  as  education  level 
(GED  =  GED  or  less  education;  COLLEGE  =  some  college),  Army  component  (GUARD 
=  National  Guard,  RESERVE  =  Army  Reserve),  and  gender  (GENDER  =  female), 
accounts  for  about  four  percent  of  the  variation  in  the  dependent  variable.  This  model  is 
statistically  significant  (%2  =  36.060,  P  <  .001),  but  its  explanatory  power  is  limited 
(Nagelkerke  R2  =  .042). 

There  is  a  statistically  significant  effect  for  the  OF  composite.  The  model  suggests 
that  having  a  higher  OF  composite  score  increases  the  odds  of  passing.  At  the  mean  OF 
score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  3%  in  the  odds  of 
passing  the  course,  and  a  five-point  increase  in  OF  would  increase  the  odds  of  passing  by 
19%.  There  are  also  significant  effects  for  Army  component,  where  Guardsmen  and 
Reservists  have  increased  odds  of  passing  (144%  and  55%,  respectively). 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

/  (1  + 

This  analysis  is  confined  to  Soldiers  from  the  Regular  Army  (though  results  would  be 
similar  for  other  demographic  groups).  Based  on  the  model,  when  the  cutoff  score  is  at 
its  current  level  (OF  =  85),  Regular  Army  Soldiers  with  an  average  OF  score  (OF  = 
106.22)  have  approximately  a  98%  chance  of  passing.  Currently,  about  97%  of  Soldiers 
are  eligible  for  MOS  88M  assignment  at  the  current  cutoff  (OF  =  85).  Lowering  the 
cutoff  by  five  points  (OF  =  80)  would  increase  eligibility  by  two  percentage  points  (to 
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99.5%)  and  the  average  Soldier  who  would  qualify  for  training  would  practically  have  the 
same  chance  of  passing  (98%).  Raising  the  cutoff  score  by  five  points  (OF  =  90),  the 
average  Soldier  would  have  the  same  chance  of  passing  (98%)  but  eligibility  would  fall 
by  about  seven  percentage  points. 


Table  2.  88M:  Probability  that  a  Soldier  (from  the  larger  Army  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model 

Cutoff  = 

75 

Cutoff = 

80 

Cutoff = 

85 

Cutoff = 

90 

Cutoff = 

95 

Percent  Eligible 
(Regular  Army) 

99.9% 

99.5% 

96.9% 

89.8% 

7478.5% 

Mean 

105.46 

105.59 

106.22 

107.71 

109.93 

Passing  rate: 

High  school  male 

98.3% 

98.3% 

98.4% 

98.4% 

98.6% 

Figure  1  shows  the  relationship  between  OF  and  the  probability  of  passing  for 
Male  Regular  Anny  Soldiers  with  a  high  school  diploma,  including  upper  and  lower 
bounds  based  upon  the  standard  error  of  the  estimated  OF  coefficient.  For  a  particular  OF 
score,  trace  a  vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to 
determine  the  corresponding  probability  of  passing.  For  example,  an  OF  score  of  100 
corresponds  to  a  passing  probability  of  about  95%  for  a  high  school  educated,  Regular 
Anny  male  Soldier.  One  can  see  that  Soldiers  with  a  relatively  low  OF  score  have  a  very 
good  chance  of  passing  the  course.  As  OF  scores  increase,  the  chance  that  a  Soldier  will 
pass  the  course  increases  slightly. 
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Chance  of  Passing 


Figure  1 :  Predicted  Probalility  of  Passing:  Male  High  School  Soldiers 


—  Regular  Army  -  -  -  Army  Upper  -  -  -  Army  Lower 
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Appendix:  Soldier  Characteristics  (88M) 


Gender 

Male 

Female 

fail 

1.3% 

1.8% 

pass 

98.7% 

98.2% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

.6% 

1 .5% 

.9% 

pass 

99.4% 

98.5% 

99.1% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

1 .2% 

.8% 

pass 

98.8% 

99.2% 
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89B  (55B):  Ammunition  Specialist 


The  final  sample  included  2680  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (98.8%)  or  failed  for  academic  reasons  (1.2%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (65%,  35%  female),  most  had  a  high  school 
diploma  but  not  more  education  (90%,  5%  some  college,  5%  GED  or  less),  and  most 
were  from  the  Regular  Army  (71%,  6%  Guard,  23%  Reserve).  The  governing  AA 
composite,  Skilled  Technical  (ST),  for  this  MOS  has  a  cutoff  score  of  92;  the  sample 
mean  is  102.54  (standard  deviation  =  10.658).  See  the  Appendix  for  a  description  of 
Soldier  characteristics  for  this  MOS  training  sample. 


Table  1.  89B:  Results  of  the  binary  logistic 
prediction  model:  ST  only  model _ 


Chi-  Square 

10.215* 

Log  Likelihood 

345.583 

Nagelkerke  R  Square 

.0 

131 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

.089 

1.093 

.003 

RESERVE 

1.018 

2.767 

2.707 

GENDER 

.098 

1.103 

.072 

ST 

.041 

1.042 

6.657** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 


A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  model,  including  the  ST  composite,  Army  component 
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(RESERVE  =  Army  Reserve),  and  gender  (GENDER  =  female),  accounts  for  about  three 
percent  of  the  variation  in  the  dependent  variable.  This  model  is  statistically  significant 
(X  =  10.215,  P  =  .017),  and  has  limited  explanatory  power  (Nagelkerke  R  =  .031). 

There  is  a  statistically  significant  effect  for  the  ST  composite.  The  model  suggests 
that  having  a  higher  ST  composite  score  increases  the  odds  of  passing.  At  the  mean  ST 
score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  four  percent  in  the 
odds  of  passing  the  course,  and  a  five-point  increase  in  ST  would  increase  the  odds  of 
passing  by  23%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  probabilities  were  calculated  using  the  formula  for  finding  probability  in  a 
binary  logistic  model,  which  is 

Q(P'X>  /  (1  +  Q(fl’x)y 

Because  GUARD,  RESERVE,  and  GENDER  were  not  statistically  significant,  this 
analysis  is  confined  to  the  modal  demographic — Regular  Army  high  school  educated 
Soldiers  factored  by  GENDER  (though  results  would  be  similar  for  any  demographic 
combination).  Based  on  the  model,  when  the  cutoff  score  is  at  its  current  level  (ST  =  92), 
male  Soldiers  with  a  high  school  diploma  and  an  average  ST  score  (ST  =  107.60)  have 
approximately  a  99%  chance  of  passing.  Currently,  about  89%  of  Soldiers  are  eligible 
for  MOS  89B  assignment  at  the  current  cutoff.  Lowering  the  ST  cutoff  by  five  points 
(ST  =  87)  would  increase  eligibility  by  eight  percentage  points,  while  the  average  male 
Soldier  who  would  qualify  for  training  would  have  essentially  the  same  chance  of  passing 
(99%).  Raising  the  ST  cutoff  scores  by  five  points  (ST  =  97),  the  average  male  Soldier 
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who  would  still  qualify  for  the  MOS  would  have  about  the  same  chance  of  passing 
(99%),  but  fewer  Soldiers  would  be  eligible  (78%). 


Table  2.  89B:  Probability  that  a  Soldier  (from  the  larger  Anny  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model 

Cutoff:  ST 
=  82 

Cutoff:  ST 
=  87 

Cutoff:  ST 
=  92 

Cutoff:  ST 
=  97 

Cutoff:  ST 
=  102 

Percent  Eligible 
( Regular  Army) 

99.8% 

97.1% 

89.0% 

78.2% 

64.9%o 

Mean:  ST 

106.79 

107.41 

109.06 

111.14 

113.60 

Passing  rates: 

High  School  Male 

98.9% 

98.9% 

99.0% 

99.0% 

99.1% 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
high  school  educated  male  Soldiers,  including  upper  and  lower  bounds  based  upon  the 
standard  error  of  the  estimated  ST  coefficient.  For  a  particular  ST  score,  trace  a  vertical 
line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  detennine  the 
corresponding  probability  of  passing.  For  example,  an  ST  value  of  100  corresponds  to  a 
passing  probability  of  about  99%  for  a  high  school  educated  male  Soldier. 
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Figure  1:  Predicted  Probability  of  Passing:  High  School  Male  Soldiers 


Appendix  A:  Soldier  Characteristics  (89B) 


Gender 

Outcome 

Male 

Female 

fail 

1.1% 

1.4% 

pass 

98.9% 

98.6% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

0.7% 

1 .3% 

0.7% 

pass 

99.3% 

98.7% 

99.3% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

1.5% 

0.5% 

0.6% 

pass 

98.5% 

99.5% 

99.4% 
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89D  (55D):  Explosive  Ordinance  Disposal  Specialists 
Phase  I  Training 

The  final  sample  included  445  soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  training  attempt  for  the  first  phase  of  this  MOS  during 
the  period  from  2001  to  2004.  Only  soldiers  who  graduated  (47.5%)  or  failed  for 
academic  reasons  (52.8%)  were  included  in  the  analysis  sample  because  we  assume  that 
non-completion  for  non-academic  reasons  is  not  related  to  cognitive  criteria. 

In  this  sample,  most  were  male  (91%,  9%  female),  most  had  a  high  school 
diploma  but  not  more  education  (87%,  13%  some  college,  <  1%  GED  or  less),  and  most 
were  Regular  Army  (98%,  2%  National  Guard).  The  governing  AA  composite,  General 
Maintenance  (GM),  for  this  MOS  has  a  cutoff  score  of  104;  the  sample  mean  is  116.13 
(standard  deviation  =  8.738).  See  the  Appendix  for  a  description  of  Soldier 
characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  for  this  MOS  and  focused  on 
the  first  training  attempt.  The  estimation  results  are  reported  in  Table  1  and  indicate  that 


Table  1.  55D:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

47.893*** 

Log  Likelihood 

567.603 

Nagelkerke  R  Square 

.136 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-9.560 

.000 

41.295*** 

COLLEGE 

-.423 

.655 

1.804 

GENDER 

.013 

1.013 

.001 

GM 

.082 

1.085 

40.769*** 

*  =  p  <  .05  **  =  p  <  .01 

***  =  p  <  .001 
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a  model  including  the  GM  composite,  education  status  (COLLEGE  =  some  college),  and 
gender  (GENDER  =  female)  accounts  for  about  14  percent  of  the  variation  in  the 
dependent  variable.  This  model  is  statistically  significant  (x  =  47.893,  P  <  .001),  and  has 
moderate  explanatory  power  (Nagelkerke  R  =  .136). 

There  is  a  statistically  significant  effect  for  the  GM  composite.  The  model 
suggests  that  having  a  higher  GM  composite  score  increases  the  odds  of  passing.  At  the 
mean  GM  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  nine 
percent  in  the  odds  of  passing  the  course,  and  a  five-point  increase  in  GM  would  increase 
the  odds  of  passing  by  51%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  soldier  from  the 
larger  Anny  applicant  population  would  pass  the  course  on  the  first  attempt  based  upon 
the  binary  logistic  model.  These  were  calculated  using  the  formula  for  finding  probability 
in  a  binary  logistic  model,  which  is 

Q(P’x)  /  (1  +  Q(fr*)y 

Because  GENDER  and  COLLEGE  were  not  statistically  significant,  this  analysis  is 
confined  to  the  modal  demographics — male  Soldiers  with  a  high  school  diploma  (though 
results  would  be  similar  for  any  demographic  combination).  Based  on  the  model,  when 
the  cutoff  score  is  at  its  current  level  (GM  =  104),  regular  Army  soldiers  with  an  average 
GM  score  (GM  =  1 14.57)  have  approximately  a  46%  chance  of  passing.  Currently,  about 
51%  of  Soldiers  are  eligible  for  MOS  55D  assignment  at  the  current  cutoff  (GM  =  104). 
Raising  the  cutoff  by  five  points  (GM  =  109),  the  average  Soldier  who  would  qualify  for 
training  would  have  a  higher  chance  of  passing  (53%),  but  eligibility  would  decrease  by 
14  percentage  points  (to  37%).  Lowering  the  cutoff  score  by  five  points  (GM  =  99) 
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would  increase  eligibility  by  13  percentage  points,  but  the  average  Soldier  would  have  an 
even  lower  chance  of  passing  (40%). 


Table  2.  55D:  Probability  that  a  male  Soldier  (from  the  larger  Army 
contract  population)  will  pass  the  course  based  on  the  binary  logistic 

model 

Cutoff  = 

Cutoff = 

Cutoff = 

Cutoff = 

Cutoff = 

94 

99 

104 

109 

114 

Percent  Eligible 
(Regular  Army) 

77.7% 

64.2% 

51.0% 

36.8% 

25.2%, 

Mean 

109.15 

111.78 

114.57 

117.88 

121.13 

Passing  rate: 

Male/High  School 

35.2% 

40.1% 

45.9% 

52.7% 

59.2% 

Figure  1  shows  the  relationship  between  GM  and  the  probability  of  passing  for 
Regular  Army  Soldiers,  including  upper  and  lower  bounds  based  upon  the  standard  error 
of  the  estimated  GM  coefficient.  For  a  particular  GM  score,  trace  a  vertical  line  up  to  the 
curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the  corresponding 
probability  of  passing.  For  example,  a  GM  score  of  1 15  corresponds  to  a  passing 
probability  of  about  47%  for  a  high  school  educated  male  Soldier. 
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d babi lity  of  passing 


Figure  1:  Predicted  Probability  of  Passing:  Male  Regular  Army 


GM  composite  score 


Testing  a  Central  Assumption  of  the  Approach 


Central  to  our  approach  has  been  the  estimation  of  model  parameters  using  only 
academic  passes  and  failure  cases  -  we  specifically  exclude  failure  cases  that  have  been 
identified  with  non-academic  reasons  for  failure.  In  doing  so  we  assume  that  non¬ 
completion  for  non-academic  reasons  is  not  related  to  cognitive  criteria.  In  this  section 
we  report  on  our  attempt  examine  the  reasonableness  of  this  assumption. 

When  we  include  non-academic  failure  cases  in  the  data  sample,  we  would  expect 
the  goodness-of-fit  to  deteriorate  and  to  see  smaller  predicted  composite  effects,  if  higher 
scoring  Soldiers  predominate  in  these  “additional”  cases.  Alternatively,  if  lower  scoring 
Soldiers  predominate  in  these  cases,  it  is  possible  to  see  little  or  no  deterioration  in  the  fit 
of  the  model.  In  Table  2a  we  present  the  regression  model  results  when  an  additional  230 
non-academic  failure  cases  are  included  in  the  analysis  sample  (for  a  total  of  675  cases). 
There  is  a  clear  deterioration  in  the  explanatory  power  of  the  model:  the  Nagelkerke  R2 
declines  from  0.136  in  the  original  sample  to  0.084  in  the  augmented  sample.  In 
addition,  the  GM  composite  effect  falls  from  1.085  to  1.066  in  the  augmented  sample. 


Table  2a.  55D:  Results  of  the  binary  logistic 
prediction  model  including  academic  failure  and 
non-academic  failure  cases 


Chi-  Square 

41.726*** 

Log  Likelihood 

795.256 

Nagelkerke  R  Square 

.084 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-8.157 

.000 

42.291*** 

COLLEGE 

-.273 

.761 

1.039 

GENDER 

-.151 

.860 

.227 

GM 

.063 

1.066 

35.418*** 

*  =  p<  .05  **  =  p<  .01 

***  =  p<  .001 
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An  even  more  extreme  comparison  might  be  between  the  original  sample  and  a 
sample  utilizing  only  the  non-academic  failure  cases  -  i.e.,  replacing  the  academic  failure 
cases  in  the  original  sample  with  non-academic  failure  cases.  The  hypothesis  is  that  the 
latter  bring  increased  randomness  to  the  data  set,  and  that  the  resulting  regression  model 
will  not  fit  as  well,  and  the  estimated  governing  composite  effects  will  be  weaker.  In 
Table  2b  we  present  the  regression  model  results  when  only  non-academic  failure  cases 
are  included  (along  with  academic  pass  cases).  This  data  sample  has  a  total  of  440 
observations.  We  find  even  greater  deterioration  in  the  explanatory  power  of  the  model: 
the  Nagelkerke  R“  in  the  “non-academic  failure”  model  is  only  about  half  that  of  the 
original  sample  model  (.059  versus  .136).  In  addition,  GM  composite  effect  is 
considerably  weaker  -  1 .047  for  the  sample  including  non-academic  failures  compared  to 
1.085  for  the  original  sample. 


Table  2b.  55D:  Results  of  the  binary  logistic 
prediction  model  including  only  non-academic 
failure  cases 


Chi-  Square 

19.812*** 

Log  Likelihood 

589.248 

Nagelkerke  R  Square 

.059 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-5.450 

.004 

14.249*** 

COLLEGE 

-.132 

.877 

.183 

GENDER 

-.339 

.713 

.948 

GM 

.046 

1.047 

14.105*** 

*  =  p<  .05  **  =  p<  .01 

***  =  p<  .001 


The  results  of  these  additional  analyses  support  the  approach  taken  to 
include  only  academic  pass  and  failure  cases  in  the  data  analysis  sample.  The  non- 
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academic  failure  cases  were  found  to  vary  in  a  less  predictable  way  with  cognitive  criteria 
and,  we  conclude,  to  represent  the  outcome  of  a  different  process. 
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Appendix:  Soldier  Characteristics  (55D):  Phase  I 


Gender 

Outcome 

Male 

Female  | 

fail 

51.9% 

62.5% 

pass 

48.1% 

37.5% 

Education  Level  j 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

50.0% 

52.8% 

52.6% 

pass 

50.0% 

47.2% 

47.4% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

53.5% 

12.5% 

pass 

46.5% 

87.5% 
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55D  (Phase  I  Training) 

Risk  Analysis  of  Policy  Alternatives 


In  this  section  we  extend  the  policy  analysis  with  a  risk  analysis  of  the  policy 
alternatives  considered  using  simulation  methods.  We  employ  the  “@RISK”  software 
package  which  is  designed  as  an  add-in  for  Microsoft  Excel.54  There  are  four  steps  in  the 
risk  analysis. 

The  first  step  is  developing  a  model.  A  regression  model  for  predicting  the 
passing  rate  has  been  specified  and  estimated,  and  the  results  stored  in  Excel  worksheet 
format.  See  Table  1. 

The  second  step  is  identifying  the  uncertainty  inherent  in  both  the  input  variables 
(composite  scores,  Soldier  characteristics)  and  the  corresponding  estimated  parameters 
and  specifying  their  possible  values  with  probability  distributions. 

To  approximate  the  governing  composite  distribution,  we  use  the  larger  Army 
contract  population  (EAF  file:  those  who  contracted  from  Jan  1992  to  Aug  2003).  This 
distribution  can  be  assumed  to  follow  a  normal  distribution.  The  mean  and  standard 
deviation  estimated  from  the  contract  population  and  used  as  input  to  the  simulations  are 
shown  in  Table  R-l  (before  truncation). 


Table  R-l.  Input  infonnation  for  the  simulations 

Mean 

Std  error  / 
deviation 

Constant 

-9.560 

1.488 

GM  var 

103.96  (before  truncation) 

12.973 

GM  coeff 

.082 

.013 

COLLEGE  var 

12.8% 

COLLEGE  coeff 

-.423 

.315 

GENDER  var 

9% 

GENDER  coeff 

.013 

.360 

To  approximate  the  distributions  for  the  demographic  variables  we  rely  on  the 
training  data  sample  used  to  estimate  the  regression  model.  These  distributions  can  be 
assumed  to  follow  binomial  distributions  with  means  estimated  from  the  training  data 
sample  as  shown  in  Table  R-l. 


54  Software  is  available  from  Palisade  Corporation,  Newfield,  NY. 
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To  approximate  the  distributions  for  the  estimated  parameters  in  the  logit  model, 
we  note  that,  under  very  general  conditions,  the  estimates  will  be  asymptotically  normally 
distributed.55  The  estimated  coefficients  (means  and  standard  errors)  are  shown  in  Table 


R-l. 


The  third  step  is  analyzing  the  model  with  simulation  to  determine  the  range  and 
probabilities  of  all  possible  training  outcome  results.  The  results  reported  represent  the 
simulation  of  the  regression  prediction  equation  for  1000  iterations  using  a  fixed  seed.56 
The  comparisons  are  between  the  baseline  case  (GM  =  104)  and  the  policy  cases  (GM  = 
99  and  GM  =  109).  These  cases  are  delineated  by  the  cutoff  score  level  which  serves  as 
the  lower  truncation  point  in  the  governing  composite  input  distribution.  The  simulation 
results  in  Table  R-2  describe  the  effect  on  the  mean  probability  of  passing  as  the  GM 
cutoff  score  level  is  varied,  and  represent  an  improvement  of  the  policy  analysis 
presented  in  Table  2  because  they  reflect  the  simultaneous  effects  of  all  known  sources  of 
uncertainty. 


Table  R-2.  55D  simulation  resu 

Its 

GM  =  99 

GM  =  104 

GM=  109 

Mean  governing 
composite 

111.36 

114.32 

117.36 

Mean  probability 
of  passing 

42.37% 

46.19% 

50.30% 

Std  deviation 

32.22% 

33.09% 

33.15% 

The  frequency  distributions  of  simulated  effects  are  shown  below  in  the  histogram 
graphs.  The  histogram  portrays  the  full  range  of  outcomes  that  average  to  the  mean 
effect  as  reported.  Looking  at  the  histogram  for  the  baseline  case  (GM  =  104), 
probability  of  passing  is  plotted  on  the  horizontal  axis  and  the  height  of  each  bar  indicates 
the  relative  number  of  Soldiers  at  that  passing  probability.  The  information  in  the 
histograms  is  effectively  summarized  in  Table  R-3.  In  that  table  we  see  the  percentage  of 
Soldiers  who  meet  or  exceed  a  given  chance  of  passing.  For  example,  looking  at  the 
baseline  case  (GM  =  104),  about  31%  of  Soldiers  have  a  70%  or  greater  chance  of 
passing. 

We  find  that  the  mean  probability  of  passing  in  the  simulation  is  46%  (Table  R-2, 
baseline).  Looking  at  the  histograms,  we  see  the  largest  portion  of  Soldiers  is  in  the 
lowest  range  of  passing  scores,  and  the  next  biggest  group  is  in  the  highest  range  of 


55  The  intuition  is  that  logit  estimates  are  derived  from  maximum  likelihood  (ML)  estimation,  and  under 
fairly  general  conditions  ML  estimates  are  consistent  and  asymptotically  normally  distributed. 

56  Latin  Hypercube  (LH)  sampling  is  utilized  in  drawing  samples  for  the  input  distributions.  It  is  designed 
to  accurately  recreate  the  input  distribution  through  sampling  in  fewer  iterations  when  compared  with  the 
Monte  Carlo  method.  The  key  to  LH  sampling  is  stratification  of  the  input  probability  distributions. 
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passing  scores.  We  note  that  the  mean  simulated  passing  probability  turns  out  to  be 
essentially  the  same  as  the  static  regression  prediction  at  the  mean  GM  score  (Table  2). 


Table  R-3. 

Percent  of  simulated  Soldiers  that  meet  or  exceed 
given  chance  of  passing 


Chance  of 
passing 

GM  =  99 

GM  =  104 

GM  =  109 

95.0% 

7.1% 

7.6% 

9.4% 

90.0% 

1 1 .6% 

14.5% 

17.6% 

85.0% 

15.1% 

19.6% 

22.8% 

80.0% 

19.3% 

23.6% 

27.7% 

22.1% 

27.2% 

32.0% 

70.0% 

24.9% 

31.3% 

36.5% 

65.0% 

28.7% 

34.6% 

40.1% 

60.0% 

33.1% 

38.1% 

43.1% 

55.0% 

35.8% 

41.5% 

46.3% 

50.0% 

39.0% 

44.8% 

49.3% 

In  the  simulated  predictions  we  are  calculating  an  average  over  all  Soldiers  and  taking 
into  account  all  sources  of  uncertainty,  whereas  in  the  static  prediction  we  are  calculating 
the  probability  of  passing  only  at  the  mean  GM  score,  ignoring  the  distribution  of  Soldier 
scores  etc.  as  well  as  uncertainty.  While  recognizing  this  variability  is  important,  we  also 
note  that  the  policy  case  impacts  relative  to  the  baseline  are  the  same  in  both  simulation 
and  static  prediction  modes. 

The  fourth  step  is  making  a  decision  based  on  the  results  provided  and  the 
preferences  of  the  school  proponent.  This  involves  the  evaluation  by  the  school 
proponent  of  the  tradeoff  between  increased  eligibility  and  reduced  passing  probabilities 
(and  vice-versa). 
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Distribution  for  PASS  EQUATION  104 


X  <=0.02  X  <=0.97 


Distribution  for  PASS  EQUATION  109 


X  <=0.03  X  <=0.98 
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Distribution  for  PASS  EQUATION  99 


X  <=0.02  X  <=0.97 
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89D  (55D):  Explosive  Ordinance  Disposal  Specialists 
Phase  II  Training 

The  final  sample  included  178  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  training  attempt  for  both  first  and  second  phases  of 
this  MOS  during  the  period  from  2001  to  2004.  Only  Soldiers  that  graduated  (33.1%)  or 
failed  for  academic  reasons  (66.9%)  were  included  in  the  analysis  sample  because  we 
assume  that  non-completion  for  non-academic  reasons  is  not  related  to  cognitive  criteria. 

In  this  sample,  most  were  male  (96%,  4%  female),  most  had  a  high  school 
diploma  but  not  more  education  (87%,  13%  some  college57),  and  most  were  Regular 
Army  (97%,  3%  National  Guard).  The  governing  AA  composite  is  GM  with  a  cutoff 
score  of  104;  in  this  sample  it  has  a  mean  of  1 18.52  (standard  deviation  =  8.704). 

A  binary  logistic  regression  model  was  estimated  for  Phase  II  training  of  this 
MOS  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  a  model  including  pass  /  fail  in  the  first  attempt  of  the  first  phase 
(S1PF1),  the  GM  composite,  education  status  (COLLEGE  =  some  college),  and  gender 
(GENDER  =  female)  accounts  for  about  four  percent  of  the  variation  in  the  dependent 
variable.  This  model  is  not  statistically  significant  (x  =  5.548,  P  =  .236),  and  its 

2  58 

explanatory  power  is  limited  (Nagelkerke  R"  =.043). 


57  One  Soldier  had  earned  a  GED  or  had  not  earned  a  high  school  diploma,  and  was  subsequently  included 
with  the  high  school  graduates  for  the  purpose  of  analysis. 

5S  In  the  Appendix  we  report  models  focused  on  the  unique  effects  of  (a)  passing  versus  failing  during  the 
first  phase,  (b)  the  GM  composite,  and  (c)  the  GT  composite.  The  first  model  is  not  statistically  significant, 
while  the  second  model  is  significant.  The  GT  models  fits  somewhat  better  than  the  GM  model,  but  not 
enough  better  to  make  a  plausible  case  for  cognitive  factors  as  decisive  in  successful  completion. 
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There  is  a  statistically  significant  effect  for  the  GM  composite.  The  model 
suggests  that  having  a  higher  GM  composite  score  increases  the  odds  of  passing.  At  the 
mean  GM  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  a  five 
percent  in  the  odds  of  passing  the  course,  and  a  five-point  increase  in  GM  would  increase 
the  odds  of  passing  by  25%. 

Policy  Analysis.  Given  that  the  Army  does  not  directly  assign  Soldiers  to  the 
second  phase  of  training,  a  separate  policy  analysis  is  not  appropriate.  Nevertheless,  it  is 
clear  that  Phase  II  pass  rates  are  low  and  that  a  model  utilizing  ASVAB  predictors  has 
limited  explanatory  power.  Presumably,  non-cognitive  factors  are  at  work. 


Table  1.  55D:  Results  of  the  binary  logistic 
prediction  model  using  pass/fail  in  the  first 
attempt  of  the  first  phase  to  predict  success  in  the 
second  phase _ _ 


Chi-  Square 

5.548 

Log  Likelihood 

220.586 

Nagelkerke  R  Square 

.043 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-6.019 

.002 

2.356** 

COLLEGE 

-.399 

.671 

.623 

GENDER 

.479 

1.614 

.380 

GM 

.045 

1.046 

5.002* 

SI  PF1 

-.037 

.964 

.009 

*=  p<.05  **  =  p  <  .01 
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Appendix:  Additional  Regression  Models  -  Phase  II 


Table  la.  55D:  Results  of  the  binary  logistic 
prediction  model  using  pass/fail  in  the  first 
attempt  of  the  first  phase  to  predict  success  in  the 
second  phase _ _ 


Chi-  Square 

.208 

Log  Likelihood 

225.926 

Nagelkerke  R  Square 

.002 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-.827 

.438 

6.656** 

SI  PF1 

.167 

1.182 

.369 

*  =  p  <  .05  **  =  p  <  .01 

***  =  D<.001 


Table  lb.  55D:  Results  of  the  binary  logistic 
prediction  model  using  the  governing  composite 
to  predict  success  in  the  second  phase 

Chi-  Square 

4.3561* 

Log  Likelihood 

221.573 

Nagelkerke  R  Square 

.035 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-5.408 

.004 

5.800* 

GM 

.040 

1.040 

4.451* 

*  =  p<  .05  **  =  p<  .01 

***  =  p<  .001 
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Table  lc.  55D:  Results  of  the  binary  logistic 
prediction  model  using  the  GT  composite  to 
predict  success  in  the  second  phase 

Chi-  Square 

6.617** 

Log  Likelihood 

219.517 

Nagelkerke  R  Square 

.051 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-6.295 

.002 

7.858 

GT 

.048 

1.049 

6.304* 

*  =  p  <  .05  **  =  p  <  .01 

***  =  p  <  .001 
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Appendix:  Soldier  Characteristics  (55D):  Phase  II 


Gender 

Outcome 

Male 

Female  [ 

fail 

67.1% 

62.5% 

pass 

32.9% 

37.5% 

Education  Level  I 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

100.0% 

66.2% 

69.6% 

pass 

.0% 

33.8% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

68.8% 

.0% 

.0% 

pass 

31.2% 

.0% 

100.0% 
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92A:  Automated  Logistics  Specialist 


The  final  sample  included  11458  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  for  this  MOS  during  a  period  from  2001 
to  2004.  Only  Soldiers  who  graduated  (96.9%)  or  failed  for  academic  reasons  (3.1%) 
were  included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (54%,  46%  female),  most  had  a  high  school 
diploma  but  not  more  education  (87%,  7%  some  college,  6%  GED  or  less),  and  most 
were  from  were  from  the  Regular  Anny  (50%,  25%  National  Guard,  25%  Reserve).  The 
governing  AA  composites,  Clerical  (CL),  for  this  MOS  has  cutoff  score  of  92;  the  sample 
mean  is  101.82  (standard  deviation  =  10.423).  See  the  Appendix  for  a  description  of 
Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  model,  including  the  CL  composite,  education  status  (GED, 
=  GED  or  less  than  a  high  school  diploma,  COLLEGE  =  some  college  education),  Anny 
component  (GUARD  =  National  Guard,  RESERVE  =  Army  Reserve),  and  gender 
(GENDER  =  Female)  accounts  for  about  seven  percent  of  the  variation  in  the  dependent 

variable.  This  model  is  statistically  significant  (x  =  197.879,  P  <  .001),  and  has 

2 

somewhat  limited  explanatory  power  (Nagelkerke  R~  =  .070). 
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Table  1.  92  A:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

197.879*** 

Log  Likelihood 

2995.211 

Nagelkerke  R  Square 

.070 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-2.118 

.120 

16.423*** 

GED 

.687 

1.987 

4.248* 

COLLEGE 

.833 

2.301 

6.668** 

GUARD 

.510 

1.665 

13.131*** 

RESERVE 

.666 

1.946 

19.297*** 

GENDER 

1.065 

2.901 

76.139*** 

CL 

.048 

1.050 

87.170*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p<  .01 


There  is  a  statistically  significant  effect  for  the  CL  composite.  The  model 
suggests  that  having  a  higher  CL  composite  score  increases  the  odds  of  passing.  At  the 
mean  CL  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  five 
percent  in  the  odds  of  passing  the  course,  and  a  five-point  increase  in  CL  would  increase 
the  odds  of  passing  by  27%.  There  are  also  statistically  significant  effects  for  GED, 
COLLEGE,  GUARD,  RESERVE,  and  GENDER,  with  increased  odds  of  passing  the 
course  at  99%,  130%,  67%,  95%,  and  190%  (for  females),  respectively. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

Q(P'x)  /  (1  + 

The  following  analysis  is  focused  upon  male  and  female  Soldiers  at  all  three  education 
levels  from  the  Regular  Army.  Based  on  the  model,  when  the  cutoff  score  is  at  its  current 
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level  (CL  =  92),  male  (female)  Soldiers  with  a  high  school  diploma  and  an  average  CL 
score  (CL  =  109.17)  have  approximately  a  96%  (99%)  chance  of  passing.  Currently, 
about  92%  of  Soldiers  are  eligible  for  MOS  92A  assignment  at  the  current  cutoff. 
Lowering  the  cutoff  by  five  points  (CL  =  87)  would  increase  eligibility  by  seven 
percentage  points  (to  99%)  and  the  average  male  (female)  Soldier  who  would  qualify  for 
training  would  essentially  the  same  chance  of  passing  (95%)  (98%).  Raising  the  CL 
cutoff  score  by  five  points  (CL  =  97)  provides  about  the  same  in  passing  rates  for  the 
average  high  school  male  (female)  Soldier  (96%)  (99%),  but  fewer  Soldiers  would  be 
eligible  (80%). 


Table  2.  92A:  Probability  that  a  Regular  Army  Soldier  (from  the  larger  successful 
applicant  population)  will  pass  the  course  based  on  the  binary  logistic  model _ 


Cutoff: 

Cutoff: 

Cutoff 

Cutoff 

Cutoff 

CL  =  82 

CL  =  87 

CL  =  92 

CL  =  97 

CL  =  102 

Percent  Eligible 
( Regular  Army) 

99.9% 

99.0% 

91.6% 

80.0% 

67.6% 

Mean 

107.49 

107.69 

109.17 

111.37 

113.62 

Passing  Rates: 

GED  Male 

97.7% 

97.7% 

97.8% 

98.0% 

98.2% 

GED  Female 

99.2% 

99.2% 

99.2% 

99.3% 

99.4% 

High  School  Male 

95.4% 

95.5% 

95.8% 

96.2% 

96.6% 

High  School  Female 

98.4% 

98.4% 

98.5% 

98.7% 

98.8% 

College  Male 

98.0% 

98.0% 

98.1% 

98.3% 

98.5% 

College  Female 

99.3% 

99.2% 

99.3% 

99.4% 

99.5% 

Figure  1  shows  the  relationship  between  CL  and  the  probability  of  passing  for 
high  school  educated  Regular  Army  Soldiers  by  gender.  For  a  particular  CL  score,  trace 
a  vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  detennine  the 
corresponding  probability  of  passing.  For  example,  a  CL  score  of  100  corresponds  to  a 
passing  probability  of  about  98%  for  a  high  school  educated,  Regular  Army  male  Soldier. 
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Figure  1:  Predicted  Probability  of  Passing:  High  School  Soldiers 


^^^High  School  Male  High  School  Female 


Figure  2  shows  the  relationship  between  CL  and  the  probability  of  passing  for 
male  Soldiers  by  education  level.  The  graph  illustrates  that  college  educated  Soldiers  and 
GED  soldiers  have  a  noticeably  better  chance  of  passing  this  course  when  compared  to 
those  with  high  school  diplomas  and  no  college  experience. 
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Figure  2:  Predicted  Probability  of  Passing:  Male  Soldiers 


■  ■  College  Male  School  Male  ““  GED  Male 


Figure  3  shows  the  relationship  between  CL  and  the  probability  of  passing  for 
Soldiers  by  education  and  gender.  The  graph  illustrates  that  women  with  some  college 
education  have  the  best  chance  at  passing  the  course,  while  high  school  educated  males, 
the  modal  students,  have  the  lowest  chance  of  passing. 
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Che 


Figure  3:  Predicted  Probability  of  Passing:  Education  Level  by  Sex 


CL  Composite  Scores 


-  -  -  College  Male 

High  School  Male 

- GED  Male 

■  -  College  Female 

- High  School  Female 

—  GED  Female 

Appendix:  Soldier  Characteristics  (92A) 


Gender 

Outcome 

Male 

Female 

fail 

4.2% 

1.9% 

pass 

95.8% 

98.1% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

1.5% 

3.4% 

1.4% 

pass 

98.5% 

96.6% 

98.6% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

4.0% 

2.1% 

2.5% 

pass 

96.0% 

97.9% 

97.5% 
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92F:  Petroleum  Supply  Specialist 


The  final  sample  included  10,491  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (98.3%)  or  failed  for  academic  reasons  (1.7%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  this  sample,  most  were  male  (81%,  19%  female),  most  had  a  high  school 
diploma  but  not  more  education  (87%,  7%  some  college,  6%  GED  or  less),  and  most 
were  from  Regular  Anny  (58%,  16%  National  Guard,  27%  Reserve).  The  governing  AA 
composites,  Operator  /  Food  (OF)  and  Clerical  (CL),  for  this  MOS  have  cutoff  scores  of 
85  and  88  respectively;  the  sample  mean  is  101.51  (standard  deviation  =  1 1.750)  for  OF, 
and  104.1 1  (standard  deviation  =  12.094)  for  CL.  The  governing  AA  composites,  OF  and 
CL,  are  correlated  at  .783  (p  <  .001).  See  the  Appendix  for  a  description  of  Soldier 
characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 


Table  1.  92F:  Results  of  the  binary  logistic 
prediction  model 

Chi-  Square 

94.193*** 

Log  Likelihood 

1677.433 

Nagelkerke  R  Square 

.058 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-2.085 

.124 

5.816* 

RESERVE 

.880 

2.411 

12.419*** 

GUARD 

.340 

1.406 

2.204 

GENDER 

.880 

2.410 

11.973** 

OF 

.043 

1.044 

10.797** 

CL 

.016 

1.016 

1.272 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .005 

[ 
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outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  a  model,  including  the  OF  and  CL  composites,  Army 
component  (GUARD  =  National  Guard;  RESERVE  =  Army  Reserve)  and  gender 
(GENDER  =  female),  accounts  for  about  six  percent  of  the  variation  in  the  dependent 
variable.  This  model  is  statistically  significant  (%2  =  94.193,  P  <  .001),  and  has  somewhat 
limited  explanatory  power  (Nagelkerke  R2  =  .058). 

There  are  statistically  significant  effects  for  the  OF  composite,  being  female,  and 
Reserve  membership.  The  model  suggests  that  having  a  higher  OF  composite  score 
predicts  increased  odds  of  passing,  but  the  marginal  effect  is  small:  at  the  mean  OF  score, 
an  increase  of  one  point  is  associated  with  an  increase  of  about  four  percent  in  the  odds  of 
passing  the  course,  and  a  five-point  increase  in  OF  would  increase  the  odds  of  passing  by 
24%. 59  There  is  also  a  noteworthy  effect  for  GENDER:  the  odds  of  passing  the  course 
for  female  Soldiers  are  141%  higher.  Finally,  there  is  a  noteworthy  effect  for  RESERVE: 
the  odds  of  a  Reservist  passing  this  training  exceed  that  of  the  average  active  duty  soldier 
by  141%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  on  the  first  attempt  based  upon 
the  binary  logistic  model.  These  were  calculated  using  the  formula  for  finding  probability 
in  a  binary  logistic  model,  which  is 

qw'x)  /  (1  +  e(P’x>). 

This  analysis  is  confined  to  the  modal  demographics  factored  by  GENDER:  male  and 
female  Soldiers  from  the  Regular  Army.  Based  on  the  model,  when  the  cutoff  score  is  at 

59  Given  the  high  correlation  between  OF  and  CL,  similar  remarks  hold  for  the  effect  of  CL.  See  the 
Appendix  for  results  of  separate  regressions  with  OF  and  CL. 
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its  current  level  (OF  =  85,  CL  =  88),  male  soldiers  with  an  average  scores  (OF  =  108.42, 
CL  =  109.32)  have  approximately  a  99%  chance  of  passing,  while  their  female 
counterparts  also  have  approximately  a  99%  chance  of  passing.  Currently,  about  98%  of 
Soldiers  are  eligible  for  MOS  92F  assignment  at  the  current  cutoff  (OF  =  85,  CL  =  88).60 
Raising  the  cutoff  by  five  points  (OF  =  90,  CL  =  93),  the  average  male  or  female  Soldier 
who  would  qualify  for  training  would  have  essentially  the  same  chance  of  passing,  while 
eligibility  would  decrease  by  8  percentage  points  (to  90%).  Lowering  the  cutoff  score  by 
five  points  (OF  =  80,  CL  =  83)  would  increase  eligibility  to  about  100%,  and  the  average 
Soldier  would  have  essentially  the  same  chance  of  passing  as  they  have  at  the  current 
cutoff  score. 


Table  2.  92F:  Probability  that  a  male  Soldier  (from  the  larger  Army  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model _ _ _ 


Cutoff  = 
75/78 

Cutoff = 
80/83 

Cutoff = 
85/88 

Cutoff = 
90/93 

Cutoff = 
95/98 

Percent  Eligible 
(Regular  Army) 

1 00% 

99.9% 

98.4% 

90.0% 

77.2% 

Mean:  OF 

105.89 

106.12 

107.01 

109.04 

111.36 

Mean:  CL 

107.27 

107.42 

108.27 

110.53 

112.81 

Passing  rates: 

Male 

98.5% 

98.5% 

98.6% 

98.8% 

98.9% 

Female 

99.4% 

99.4% 

99.4% 

99.5% 

99.5% 

Figure  1  shows  the  relationship  between  OF  and  the  probability  of  passing  for 

male  and  female  Regular  Army  Soldiers,  based  upon  a  regression  model  that  excludes  CL 

(see  the  Appendix)  because  it  is  not  significant  in  the  reported  model.  Figure  1  includes 

upper  and  lower  bounds  based  upon  the  standard  error  of  the  estimated  OF  coefficient. 

For  a  particular  OF  score,  trace  a  vertical  line  up  to  the  curve,  and  then  a  horizontal  line 

60  The  percentage  eligible  reported  in  the  text  and  Table  2  reflect  only  the  CL  cutoff  and  not  the  joint  CL 
and  OF  cutoff  effects.  Given  the  “dominance”  of  the  CL  over  the  OF  cutoff  scores,  this  approximation  is 
fully  justified. 
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over  to  the  axis  to  determine  the  corresponding  probability  of  passing.  For  example,  an 
OF  score  of  100  corresponds  to  a  passing  probability  of  about  98%  for  a  high  school 
educated,  Regular  Army  male  Soldier.  One  can  see  that  male  Soldiers  with  a  low  OF 
score  stand  a  reasonably  good  chance  of  passing  the  course;  female  Soldiers  have  an  even 
better  chance  of  passing.  As  the  OF  (or  CL)  composite  scores  increase,  the  chance  that  a 
Soldier  will  pass  the  course  increases,  essentially  reaching  100%  just  above  the  current 
cutoff. 


Figure  1:  Predicted  Probability  of  Passing:  Male  and  Female  Students 


Male  —  -  Male  Upper  —  -  Male  Lower  ^^^Female  -  -  -  Female  Lower  -  -  -  Female  Lower 
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Appendix:  Additional  Regression  Models 


Table  1A.  92F:  Results  of  the  binary  logistic 
prediction  model  excluding  CL 

Chi-  Square 

92.879*** 

Log  Likelihood 

1678.747 

Nagelkerke  R  Square 

.057 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-1.663 

.190 

4.638* 

RESERVE 

.896 

2.449 

12.884*** 

GUARD 

.329 

1.390 

2.067 

GENDER 

.910 

2.484 

12.954*** 

OF 

.055 

1.057 

46.507*** 

*  =  p  <  .05  ***  =  p  <  .00] 

**  =  p  <  .005 

[ 

Table  IB.  92F:  Results  of  the  binary  logistic 
prediction  model  excluding  OF 

Chi-  Square 

82.986*** 

Log  Likelihood 

1688.640 

Nagelkerke  R  Square 

.051 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-1.590 

.204 

3.510 

RESERVE 

.914 

2.495 

13.478*** 

GUARD 

.401 

1.493 

3.078 

GENDER 

.748 

2.113 

8.854** 

CL 

.053 

1.054 

37.875*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .005 

[ 

384 


Appendix:  Soldier  Characteristics  (92F) 


Gender 

Observed 

Male 

Female  [1 

fail 

1.8% 

.9% 

pass 

98.2% 

99.1% 

Education  Level  I 

Observed 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

1 .8% 

.5% 

pass 

‘1 

98.2% 

99.5% 

Component  I 

Observed 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

2.2% 

.7% 

pass 

97.8% 

99.3% 
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92G:  Food  Service  Operations 


The  final  sample  included  10,206  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (99.6%)  or  failed  for  academic  reasons  (0.4%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  this  sample,  most  of  the  Soldiers  were  male  (63%,  37%  female),  most  had  a 
high  school  diploma  but  not  more  education  (90%,  6%  some  college,  4%  GED  or  less), 
and  most  were  from  Regular  Army  (65%,  19%  National  Guard,  16%  Army  Reserve).  The 
governing  AA  composite,  Operator  /  Food  (OF),  for  this  MOS  has  a  cutoff  score  of  85; 
the  sample  mean  is  98.78  (standard  deviation  =  1 1.446).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  model,  including  the  OF  composite  and  gender  (GENDER  = 


Table  1.  92G:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

12.916** 

Log  Likelihood 

499.174 

Nagelkerke  R  Square 

.026 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

.570 

1.768 

.125 

OF 

.049 

1.050 

8.484** 

GENDER 

1.003 

2.727 

6.547* 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .005 
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female),  accounts  for  about  three  percent  of  the  variation  in  the  dependent  variable.  This 
model  is  statistically  significant  (%2  =  12.916,  P  =  .002),  but  its  explanatory  power  is 
limited  (Nagelkerke  R2  =  .026). 

There  is  a  statistically  significant  effect  for  the  OF  composite.  The  model  suggests 
that  having  a  higher  OF  composite  score  increases  the  odds  of  passing.  At  the  mean  OF 
score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  5%  in  the  odds  of 
passing  the  course,  and  a  five-point  increase  in  OF  would  increase  the  odds  of  passing  by 
28%.  There  is  also  a  significant  effect  for  GENDER,  where  women  have  173%  higher 
odds  of  passing  than  men. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  on  the  first  attempt  based  upon 
the  binary  logistic  model.  These  were  calculated  using  the  formula  for  finding  probability 
in  a  binary  logistic  model,  which  is 

Q(P’x>  /  (1  + 

Based  on  the  model,  when  the  cutoff  score  is  at  its  current  level  (OF  =  85),  male  Soldiers 
with  an  average  OF  score  (OF  =  106.76)  have  approximately  a  99.7%  chance  of  passing, 
while  female  Soldiers  have  a  99.9%  chance  of  passing.  Currently,  about  97%  of  Soldiers 
are  eligible  for  MOS  92G  assignment  at  the  current  cutoff  (OF  =  85).  Lowering  the 
cutoff  by  five  points  (OF  =  80)  would  increase  eligibility  by  two  percentage  points  (to 
99%)  and  the  average  Soldier  would  have  the  same  chance  of  passing.  Raising  the  cutoff 
score  by  five  points  (OF  =  90),  the  average  Soldier  would  have  the  same  chance  of 
passing  but  eligibility  would  fall  by  about  7  percentage  points  (90%). 
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Table  2.  92G:  Probability  that  a  Soldier  (from  the  larger  Army  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model _ _ _ 


Cutoff  = 

Cutoff = 

Cutoff = 

Cutoff = 

Cutoff = 

75 

80 

85 

90 

95 

Percent  Eligible 
(Regular  Army) 

99.9% 

99.3% 

97.2% 

91.9% 

80.9% 

Mean 

105.89 

106.10 

106.76 

108.05 

110.08 

Passing  rates: 

High  School  Male 

99.7% 

99.7% 

99.7% 

99.7% 

99.7% 

High  School  Female 

99.9% 

99.9% 

99.9% 

99.9% 

99.9% 

Figure  1  shows  the  relationship  between  OF  and  the  probability  of  passing  on  the 
first  attempt  for  male  and  female  Soldiers,  including  upper  and  lower  bounds  based  upon 
the  standard  error  of  the  estimated  OF  coefficient.  For  a  particular  OF  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
corresponding  probability  of  passing.  For  example,  an  OF  score  of  100  corresponds  to  a 
passing  probability  of  about  100%  for  a  high  school  educated,  Regular  Anny  male 
Soldier.  One  can  see  that  Soldiers  with  a  low  OF  score  have  a  reasonable  chance  of 
passing  the  course.  As  OF  scores  increase,  the  chance  that  a  Soldier  will  pass  the  course 
increases. 
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Probability 


Figure  1:  Predicted  Probability  of  Passing:  Male  vs.  Female 


Appendix:  Soldier  Characteristics  (92G) 


]  Gender  1 

Outcome 

Regular  or 
Reserve 

Guard 

fail 

.5% 

.2% 

pass 

99.5% 

99.8% 

Education  Level  1 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

.2% 

.4% 

.3% 

pass 

99.8% 

99.6% 

99.7% 

Component  1 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

.5% 

.3% 

.2% 

pass 

99.5% 

99.7% 

99.8% 

390 


92 Y:  Unit  Supply  Specialist 


The  final  sample  included  9970  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  for  this  MOS  during  a  period  from  2001 
to  2004.  Only  Soldiers  who  graduated  (98.8%)  or  failed  for  academic  reasons  (1.2%) 
were  included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (56%,  44%  female),  most  had  a  high  school 
diploma  but  not  more  education  (87%,  7%  some  college,  6%  GED  or  less),  and  most 
were  from  were  from  the  Regular  Anny  (62%,  23%  National  Guard,  15%  Reserve).  The 
governing  AA  composites,  Clerical  (CL),  for  this  MOS  has  cutoff  score  of  92;  the  sample 
mean  is  102.07  (standard  deviation  =  10.587).  See  the  Appendix  for  a  description  of 
Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  model,  including  the  CL  composite,  education  status  (GED, 
=  GED  or  less  than  a  high  school  diploma,  COLLEGE  =  some  college  education),  Anny 
component  (GUARD  =  National  Guard,  RESERVE  =  Army  Reserve),  and  gender 
(GENDER  =  Female)  accounts  for  about  six  percent  of  the  variation  in  the  dependent 

variable.  This  model  is  statistically  significant  (x  =  76.078,  P  <  .001),  and  has  somewhat 

2 

limited  explanatory  power  (Nagelkerke  R“  =  .061). 
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Table  1.  92  Y:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

76.078*** 

Log  Likelihood 

1240.832 

Nagelkerke  R  Square 

.061 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-.429 

.651 

.333 

GED 

1.147 

3.149 

2.256 

COLLEGE 

.822 

2.276 

1.950 

GUARD 

.860 

2.363 

9.621** 

RESERVE 

1.127 

3.085 

8.164** 

GENDER 

1.080 

2.946 

25.761*** 

CL 

.041 

1.042 

31.075*** 

*  =  p  <  .05  ***=p<.001 

**  =  p<  .01 


There  is  a  statistically  significant  effect  for  the  CL  composite.  The  model 
suggests  that  having  a  higher  CL  composite  score  increases  the  odds  of  passing.  At  the 
mean  CL  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  four 
percent  in  the  odds  of  passing  the  course,  and  a  five-point  increase  in  CL  would  increase 
the  odds  of  passing  by  23%.  There  are  also  statistically  significant  effects  for  GUARD, 
RESERVE,  and  GENDER,  with  increased  odds  of  passing  the  course  at  136%,  209%, 
and  195%  (for  females),  respectively. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

Q(P'x>  /  (1  + 

The  following  analysis  is  focused  upon  male  and  female  Soldiers  from  the  Regular  Army. 
Based  on  the  model,  when  the  cutoff  score  is  at  its  current  level  (CL  =  92),  male  (female) 
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Soldiers  with  a  high  school  diploma  and  an  average  CL  score  (CL  =  109. 17)  have 
approximately  a  98%  (99%)  chance  of  passing.  About  92%  of  Soldiers  are  eligible  for 
MOS  92Y  assignment  at  the  current  cutoff.  Lowering  the  cutoff  by  five  points  (CL  =  87) 
would  increase  eligibility  by  seven  percentage  points  (to  99%)  and  the  average  male 
(female)  Soldier  who  would  qualify  for  training  would  have  essentially  the  same  chance 
of  passing  (98%)  (99%).  Raising  the  CL  cutoff  score  by  five  points  (CL  =  97)  provides 
about  the  same  in  passing  rates  for  the  average  high  school  male  (female)  Soldier  (98%) 
(99%),  but  fewer  Soldiers  would  be  eligible  (80%). 


Table  2.  92Y:  Probability  that  a  Regular  Army  Soldier  (from  the  larger  successful 
applicant  population)  will  pass  the  course  based  on  the  binary  logistic  model 

Cutoff: 

CL  =  82 

Cutoff: 

CL  =  87 

Cutoff 

CL  =  92 

Cutoff 

CL  =  97 

Cutoff 

CL  =  102 

Percent  Eligible 
(Regular  Army) 

99.9% 

99.0% 

91.6% 

80.0% 

67.6%o 

Mean 

107.49 

107.69 

109.17 

111.37 

113.62 

Passing  Rates: 

High  School  Male 

mmm 

mmm-. 

mmm 

High  School  Female 

mmm 

mmm 

mmm. 

» 

Figure  1  shows  the  relationship  between  CL  and  the  probability  of  passing  for 
high  school  educated  Regular  Army  Soldiers  by  gender.  For  a  particular  CL  score,  trace 
a  vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  detennine  the 
corresponding  probability  of  passing.  For  example,  a  CL  score  of  100  corresponds  to  a 
passing  probability  of  about  98%  for  a  high  school  educated,  Regular  Army  male  Soldier. 
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Chance  of  Passing 


Figure  1:  Predicted  Probability  of  Passing:  High  School  Soldiers 


CL  Composite  Scores 


-  -  -  Male  Upper  -  -  -  Male  Lower  Female  —  -  Female  Upper  —  -  Female  Lower 
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Appendix:  Soldier  Characteristics  (92Y) 


Gender 

Outcome 

Male 

Female 

fail 

1.6% 

0.7% 

pass 

98.4% 

99.3% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

0.4% 

1 .3% 

0.4% 

pass 

99.6% 

98.7% 

99.6% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

1.6% 

0.5% 

0.7% 

pass 

98.4% 

99.5% 

99.3% 
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35E:  Radio/COMSEC  Repairer 


The  final  sample  included  1268  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (93.0%)  or  failed  for  academic  reasons  (7.0%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (87%,  13%  female),  most  had  a  high  school 
diploma  but  not  more  education  (82%,  15%  some  college,  3%  GED  or  less),  and  most 
were  Regular  Army  (76%,  9%  Anny  Reserve,  14%  National  Guard).  The  governing  AA 
composite,  Electronics  Repair  (EL),  for  this  MOS  has  a  cutoff  score  of  99;  the  sample 
mean  is  1 12.21  (standard  deviation  =  10.159).  See  the  Appendix  for  a  description  of 
Soldier  characteristics  for  this  MOS  training  sample. 


Table  1.  35E:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

27.995*** 

Log  Likelihood 

616.474 

Nagelkerke  R  Square 

.0 

155 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-2.097 

.123 

2.812 

COLLEGE 

.389 

1.475 

1.136 

GUARD 

1.410 

4.096 

6.530** 

RESERVE 

.172 

1.187 

.174 

GENDER 

.865 

2.375 

3.936* 

EL 

.040 

1.041 

12.286*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 


A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
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Table  1  and  indicate  that  the  model,  including  the  EL  composite,  education  status 
(COLLEGE  =  some  college),  Anny  component  (GUARD  =  National  Guard;  RESERVE 
=  Army  Reserve)  and  gender  (GENDER  =  female),  accounts  for  about  6%  of  the 
variation  in  the  dependent  variable.  This  model  is  statistically  significant  (x  =  27.995,  P 
<  .001),  but  has  limited  explanatory  power  (Nagelkerke  R  =  .055). 

There  are  statistically  significant  effects  for  the  EL  composite,  having  some 
college  education,  and  National  Guard  membership.  The  model  suggests  that  having  a 
higher  EL  composite  score  increases  the  odds  of  passing.  At  the  mean  EL  score,  an 
increase  of  one  point  is  associated  with  an  increase  of  about  four  percent  in  the  odds  of 
passing  the  course,  and  a  five-point  increase  in  EL  would  increase  the  odds  of  passing  by 
22%.  There  is  a  noteworthy  effect  for  GUARD,  where  the  odds  of  passing  for  a  National 
Guardsman  exceed  those  of  a  Regular  Anny  Soldier  by  310%.  There  is  also  an  effect  for 
GENDER,  where  the  odds  of  a  female  Soldier  passing  this  training  exceed  that  of  the 
average  male  Soldier  by  over  138%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  fonnula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

Q(P’x)  /  (1  +  Q(P’x)y 
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Because  GED  and  COLLEGE  were  not  statistically  significant,  this  analysis  is  confined 
to  the  modal  demographics  factored  by  GENDER — high  school  educated  Soldiers  from 
the  Regular  Army  (though  results  would  be  similar  for  any  demographic  combination). 
Based  on  the  model,  when  the  cutoff  score  is  at  its  current  level  (EL  =  99),  male  Soldiers 
with  a  high  school  diploma  and  an  average  EL  score  (EL  =  1 12.02)  have  approximately  a 
92%  chance  of  passing,  while  their  female  counterparts  have  approximately  a  96% 
chance  of  passing.  Currently,  about  68%  of  Soldiers  are  eligible  for  MOS  35E 
assignment  at  the  current  cutoff  (EL  =  99).  Lowering  the  cutoff  by  five  points  (EL  =  94) 
would  increase  eligibility  by  13  percentage  points,  while  the  average  Soldier  who  would 
qualify  for  training  would  have  about  the  same  chance  of  passing  (male  =  91%,  female  = 
96%).  Raising  the  cutoff  score  by  five  points  (EL  =  104),  the  average  Soldier  who  would 
still  qualify  for  the  MOS  would  have  about  the  same  chance  of  passing  (male  =  92%, 
female  =  97%),  but  fewer  Soldiers  would  be  eligible  (53%). 


Table  2.  35E:  Probability  that  a  Soldier  (from  the  larger  Army  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model _ _ _ 


Cutoff  = 

Cutoff = 

Cutoff = 

Cutoff = 

Cutoff = 

89 

94 

99 

104 

109 

Percent  Eligible 
(Regular  Army) 

91.7% 

81.2% 

67.8% 

53.0% 

39.8% 

Mean 

107.30 

109.40 

112.02 

115.03 

118.04 

Passing  rates: 

High  school  male 

90.0% 

90.7% 

91.6% 

92.4% 

93.2% 

High  school  female 

95.5% 

95.9% 

96.3% 

96.7% 

97.0% 

Figure  1  shows  the  relationship  between  EL  and  the  probability  of  passing  for 
Regular  Army  Soldiers,  including  upper  and  lower  bounds  based  upon  the  standard  error 
of  the  estimated  EL  coefficient.  For  a  particular  EL  score,  trace  a  vertical  line  up  to  the 
curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the  corresponding 


398 


probability  of  passing.  For  example,  an  EL  value  of  100  corresponds  to  a  passing 
probability  of  about  87%  for  a  high  school  educated  male  Soldier,  and  94%  for  a  high 
school  educated  female  soldier.  One  can  see  that  the  probability  of  passing  increases 
gradually  over  the  full  range  for  Soldiers  of  both  genders. 


Figure  1 :  Predicted  Probalility  of  Passing:  High  School  Soldiers 


—  Male  -  -  -  Male  Upper  -  -  ■  Male  Lower  ——  Female  — —  Female  Upper  -  -  Female  Lower 


399 


Appendix:  Soldier  Characteristics  (35E) 


Gender 

Outcome 

Male 

Female 

fail 

7.5% 

3.8% 

pass 

92.5% 

96.2% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

.0% 

7.7% 

4.8% 

pass 

100.0% 

92.3% 

95.2% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

8.0% 

5.9% 

2.2% 

pass 

92.0% 

94.1% 

97.8% 
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35F:  Special  Electronic  Device  Repairer 


The  final  sample  included  319  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (92.8%)  or  failed  for  academic  reasons  (7.2%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  of  the  Soldiers  were  male  (92%,  8%  female),  most  had  a  high 
school  diploma  but  not  more  education  (88%,  8%  some  college,  4%  GED  or  less),  and 
most  were  from  Regular  Army  (73%,  20%  National  Guard,  6%  Anny  Reserve).  The 
governing  AA  composite,  Electrical  Repair  (EL),  for  this  MOS  has  a  cutoff  score  of  102; 
the  sample  mean  for  EL  is  1 12.84  (standard  deviation  =  8.444).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 


Table  1.  35F:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

4.494 

Log  Likelihood 

160.773 

Nagelkerke  R  Square 

.0 

135 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-1.568 

.634 

.226 

GED 

-.950 

.387 

1.292 

COLLEGE 

-.820 

.441 

1.444 

GUARD 

.267 

1.307 

.200 

GENDER 

-2.80 

.755 

.170 

EL 

.038 

1.039 

1.639 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 


A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
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Table  1  and  indicate  that  a  model,  including  the  EL  composite,  education  status  (GED  = 
GED  or  less  education;  COLLEGE  =  some  college  education),  Army  component 
(GUARD  =  National  Guard),  and  gender  (GENDER  =  female),  accounts  for  about  four 
percent  of  the  variation  in  the  dependent  variable.  This  model  is  not  statistically 
significant  (%2  =  4.494,  P  =  .481),  and  it  has  limited  explanatory  power  (Nagelkerke  R2  = 
.035). 

There  is  no  statistically  significant  effect  for  the  EL  composites  or  any  other 
variable.  The  model  suggests  that  the  EL  composite  score  does  not  predict  increased 
odds  of  passing,  but  there  are  relatively  few  failure  observations  for  a  definitive 
conclusion. 

Policy  Analysis.  Given  the  model’s  lack  of  statistical  significance  and 
explanatory  power,  this  model  is  not  appropriate  for  conducting  policy  analysis. 
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Appendix:  Soldier  Characteristics  (35F) 


Gender 

Outcome 

Male 

Female  [! 

fail 

6.8% 

ii.i% 

pass 

93.2% 

88.9% 

Education  Level  j 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

14.3% 

6.4% 

12.5% 

pass 

85.7% 

93.6% 

87.5% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

8.1% 

.0% 

pass 

91.9% 

100.0% 
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35M:  Radar  Repairer 


The  final  sample  included  245  soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  soldiers  who  graduated  (96.3%)  or  failed  for  academic  reasons  (3.7%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (93%,  7%  female),  most  had  a  high  school 
diploma  but  not  more  education  (89%,  8%  some  college,  3%  GED  or  less),  and  most 
were  Regular  Army  (80%,  2%  Anny  Reserve,  18%  National  Guard).  The  governing  AA 
composite,  Skilled  Technical  (EL),  for  this  MOS  has  a  cutoff  score  of  107;  the  sample 
mean  is  1 16.98  (standard  deviation  =  6.900).  See  the  Appendix  for  a  description  of 
Soldier  characteristics  for  this  MOS  training  sample. 


Table  1.  35M:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

12.305** 

Log  Likelihood 

64.838 

Nagelkerke  R  Square 

.181 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-17.280 

.000 

4.725* 

GUARD 

-1.436 

.238 

3.950* 

EL 

.184 

1.202 

6.694** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 


A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  model,  including  the  EL  composite  and  Army  component 
(GUARD  =  National  Guard),  accounts  for  about  18%  of  the  variation  in  the  dependent 
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variable.  This  model  is  statistically  significant  (x  =  12.305,  P  =  .010),  with  moderate 
explanatory  power  (Nagelkerke  R“ = . 1 8 1)  . 

There  are  statistically  significant  effects  for  the  EL  composite  and  National  Guard 
membership.  The  model  suggests  that  having  a  higher  EL  composite  score  increases  the 
odds  of  passing.  At  the  mean  EL  score,  an  increase  of  one  point  is  associated  with  an 
increase  of  about  20%  in  the  odds  of  passing  the  course,  and  a  five-point  increase  in  EL 
would  increase  the  odds  of  passing  by  151%.  There  is  a  noteworthy  effect  for  GUARD, 
where  the  odds  of  a  National  Guardsman  passing  this  training  are  76%  lower  than  that  of 
a  Regular  Army  Soldier. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  fonnula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

qw'x)  /  (1  +  q(P’x>). 

This  analysis  is  confined  to  the  modal  demographics — Soldiers  from  the  Regular  Army 
(though  results  would  be  similar  for  any  demographic  combination  except  National 
Guardsmen).  Based  on  the  model,  when  the  cutoff  score  is  at  its  current  level  (EL  =  107), 
Soldiers  with  an  average  EL  score  (EL  =  1 16.86)  have  approximately  a  99%  chance  of 
passing.  Currently,  about  45%  of  soldiers  are  eligible  for  MOS  35M  assignment  at  the 
current  cutoff  (EL  =  107).  Lowering  the  cutoff  by  five  points  (EL  =  102)  would  increase 
eligibility  by  15  percentage  points,  while  eligible  Soldiers  with  an  average  EL  score  (EL 
=  1 13.67)  would  have  a  slightly  lower  chance  of  passing  (97%).  Raising  the  cutoff  score 

61  A  caveat  to  the  reliability  of  this  model  is  the  fact  that  the  training  sample  contains  only  nine  failure 
cases. 
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by  five  points  (EL  =  1 12),  eligible  Soldiers  with  an  average  EL  score  (EL  =  120.29) 
would  have  essentially  the  same  chance  of  passing  (99%),  but  fewer  Soldiers  would  be 
eligible  (31%). 


Table  2.  35M:  Probability  that  a  Soldier  (from  the  larger  Army  contract  po 
with  mean  EL  score  will  pass  the  course  based  on  the  binary  logistic  mode 

pulation) 

Cutoff  = 

97 

Cutoff = 
102 

Cutoff = 
107 

Cutoff = 
112 

Cutoff  = 
117 

Percent  Eligible 
( Regular  Army) 

73.9% 

59.5% 

44. 7%0 

31.1% 

20.0% 

Mean 

110.81 

113.67 

116.86 

120.29 

123.76 

Passing  rates: 

Regular  Army 

95.7% 

97.4% 

98.6% 

99.2% 

99.6% 

Figure  1  shows  the  relationship  between  EL  and  the  probability  of  passing  for 
Regular  Army  Soldiers,  including  upper  and  lower  bounds  based  upon  the  standard  error 
of  the  estimated  EL  coefficient.  For  a  particular  EL  score,  trace  a  vertical  line  up  to  the 
curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the  corresponding 
probability  of  passing.  For  example,  an  EL  value  of  100  corresponds  to  a  passing 
probability  of  about  75%  for  a  Regular  Anny  Soldier.  One  can  see  that  the  probability  of 
passing  increases  steeply  over  the  full  range  of  the  data. 
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Figure  1:  Predicted  Probalility  of  Passing:  Regular  Army  Soldiers 


EL  Composite  Scores 


Appendix:  Soldier  Characteristics  (35M) 


Gender 

Outcome 

Male 

Female 

fail 

4.0% 

.0% 

pass 

96.0% 

100.0% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

.0% 

4.1% 

.0% 

100.0% 

95.9% 

100.0% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

2.5% 

.0% 

9.1% 

pass 

97.5% 

100.0% 

90.9% 
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35M  Risk  Analysis  of  Policy  Alternatives 


In  this  section  we  extend  the  policy  analysis  with  a  risk  analysis  of  the  policy 
alternatives  considered  using  simulation  methods.  We  employ  the  “@RISK”  software 
package  which  is  designed  as  an  add-in  for  Microsoft  Excel.  There  are  four  steps  in  the 
risk  analysis. 

The  first  step  is  developing  a  model.  A  regression  model  for  predicting  the 
passing  rate  has  been  specified  and  estimated,  and  the  results  stored  in  Excel  worksheet 
format.  See  Table  1. 

The  second  step  is  identifying  the  uncertainty  inherent  in  both  the  input  variables 
(composite  scores,  Soldier  characteristics)  and  the  corresponding  estimated  parameters 
and  specifying  their  possible  values  with  probability  distributions. 

To  approximate  the  governing  composite  distribution,  we  use  the  larger  Army 
contract  population  (EAF  file:  those  who  contracted  from  Jan  1992  to  Aug  2003).  This 
distribution  can  be  assumed  to  follow  a  normal  distribution.  The  mean  and  standard 
deviation  estimated  from  the  contract  population  and  used  as  input  to  the  simulations  are 
shown  in  Table  R-l  (before  truncation). 


Table  R-l.  Input  infonnation  for  the  simulations 

Mean 

Std  error  / 
deviation 

Constant 

-17.280 

7.950 

EL  var 

105.33  (before  truncation) 

12.525 

EL  coeff 

.184 

.071 

GUARD  var 

18.0% 

GUARD  coeff 

-1.436 

.722 

To  approximate  the  distributions  for  the  demographic  variables  (GUARD)  we  rely 
on  the  training  data  sample  used  to  estimate  the  regression  model.  These  distributions 
can  be  assumed  to  follow  binomial  distributions  with  means  estimated  from  the  training 
data  sample  as  shown  in  Table  R-l. 

To  approximate  the  distributions  for  the  estimated  parameters  in  the  logit  model, 
we  note  that,  under  very  general  conditions,  the  estimates  will  be  asymptotically  normally 
distributed.63  The  estimated  coefficients  (means  and  standard  errors)  are  shown  in  Table 
R-l. 


62  Software  is  available  from  Palisade  Corporation,  Newfield,  NY. 

63  The  intuition  is  that  logit  estimates  are  derived  from  maximum  likelihood  (ML)  estimation,  and  under 
fairly  general  conditions  ML  estimates  are  consistent  and  asymptotically  normally  distributed. 
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The  third  step  is  analyzing  the  model  with  simulation  to  determine  the  range  and 
probabilities  of  all  possible  training  outcome  results.  The  results  reported  represent  the 
simulation  of  the  regression  prediction  equation  for  1000  iterations  using  a  fixed  seed.64 
The  comparisons  are  between  the  baseline  case  (EL  =  107)  and  the  policy  cases  (EL  = 
102  and  EL  =  112).  These  cases  are  delineated  by  the  cutoff  score  level  which  serves  as 
the  lower  truncation  point  in  the  governing  composite  input  distribution.  The  results  in 
Table  R-2  describe  the  effect  on  the  mean  probability  of  passing  as  the  EL  cutoff  score 
level  is  varied,  and  represent  an  improvement  of  the  policy  analysis  presented  in  Table  2 
because  the  simulation  results  reflect  the  simultaneous  effects  of  all  known  sources  of 
uncertainty. 


Table  R-2.  35M  simulation  results 


EL  =  102 

EL  =  107 

EL  =  112 

Mean  governing 
composite 

113.01 

116.12 

119.63 

Mean  probability 
of  passing 

61.0% 

63.3% 

66.0% 

Std  deviation 

45.4% 

44.5% 

44.3% 

The  frequency  distributions  of  simulated  effects  are  shown  below  in  the  histogram 
graphs.  The  histogram  portrays  the  full  range  of  outcomes  that  average  to  the  mean 
effect  as  reported.  Looking  at  the  histogram  for  the  baseline  case  (EL  =  107),  probability 
of  passing  is  plotted  on  the  horizontal  axis  and  the  height  of  each  bar  indicates  the 
relative  number  of  Soldiers  at  that  passing  probability.  The  information  in  the  histograms 
is  effectively  summarized  in  Table  R-3.  In  that  table  we  see  the  percentage  of  Soldiers 
who  meet  or  exceed  a  given  chance  of  passing.  Lor  example,  looking  at  the  baseline  case 
(EL  =  107),  about  61%  of  Soldiers  have  a  70%  or  greater  chance  of  passing. 

We  find  that  the  mean  probability  of  passing  in  the  simulation  is  63%  (Table  R-2, 
baseline).  Looking  at  the  histograms,  we  see  a  large  portion  of  Soldiers  in  the  highest 
range  of  passing  scores;  however,  it  is  also  evident  that  a  not  insubstantial  group  of 
Soldiers  has  a  very  low  chance  of  passing.  Sensitivity  analysis  indicates  that  we  can 
attribute  this  pattern  to  the  relatively  large  standard  errors  on  the  constant  and  governing 
composite  terms  (especially  the  former)  that  come  into  play  in  the  simulation.  The 


64  Latin  Hypercube  (LH)  sampling  is  utilized  in  drawing  samples  for  the  input  distributions.  It  is  designed 
to  accurately  recreate  the  input  distribution  through  sampling  in  fewer  iterations  when  compared  with  the 
Monte  Carlo  method.  The  key  to  LH  sampling  is  stratification  of  the  input  probability  distributions. 
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Table  R-3. 

Percent  of  simulated  Soldiers  that  meet  or  exceed 
given  chance  of  passing 


Chance  of 
passing 

EL  =  102 

EL  =  107 

EL  =  112 

95.0% 

52.3% 

52.3% 

57.2% 

90.0% 

54.8% 

55.4% 

59.8% 

85.0% 

55.6% 

57.1% 

61.1% 

80.0% 

56.6% 

58.2% 

61.9% 

75.0% 

57.3% 

59.7% 

62.7% 

70.0% 

57.8% 

60.7% 

63.5% 

65.0% 

58.7% 

61.7% 

64.6% 

60.0% 

59.2% 

62.6% 

65.3% 

55.0% 

60.2% 

63.4% 

66.0% 

50.0% 

60.8% 

63.9% 

66.5% 

relatively  large  standard  errors  introduce  variability  into  the  passing  probabilities65.  We 
note  that  the  mean  simulated  passing  probability  turns  out  to  be  substantially  lower  than 
the  static  prediction  at  the  mean  EL  score  (Table  2).  In  the  simulated  predictions  we  are 
calculating  an  average  over  all  Soldiers  and  taking  into  account  all  sources  of  uncertainty, 
whereas  in  the  static  prediction  we  are  calculating  the  probability  of  passing  only  at  the 
mean  EL  score,  ignoring  the  distribution  of  Soldier  scores  etc.  as  well  as  uncertainty. 
While  recognizing  this  variability  is  important,  we  also  note  that  the  policy  case  impacts 
relative  to  the  baseline  are  the  same  in  both  simulation  and  static  prediction  modes. 

The  fourth  step  is  making  a  decision  based  on  the  results  provided  and  the 
preferences  of  the  school  proponent.  This  involves  the  evaluation  by  the  school 
proponent  of  the  tradeoff  between  increased  eligibility  and  reduced  passing  probabilities 
(and  vice-versa). 


65  The  large  standard  error  for  the  constant  allows  scores  to  be  more  spread  out  and  allows  for  large 
groupings  at  the  highest  and  lowest  values.  And  since  the  probabilities  of  passing  are  calculated  from  the 
ratio:  e^' x>  /  (1  +  XJ),  as  the  scores  calculated  from  the  equation  become  extreme,  the  ratios  are  more 
likely  to  approach  zero  and  one. 
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27M:  Multiple  Launch  Rocket  System  Repairer 


The  final  sample  included  244  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (82.8%)  or  failed  for  academic  reasons  (17.2%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  this  sample,  a  majority  of  the  students  were  male  (77%,  23%  female),  most  had 
a  high  school  diploma  but  not  more  education  (89%,  6%  some  college,  5%  GED  or  less), 
and  the  greatest  number  were  from  the  Regular  Army  (68%,  32%  National  Guard).  The 
governing  AA  composite,  Electrical  Repair  (EL),  for  this  MOS  has  a  cutoff  score  of  95; 
the  sample  mean  is  103.42  (standard  deviation  =  9.199).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 


Table  1.  27M:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

25.393*** 

Log  Likelihood 

198.721 

Nagelkerke  R  Square 

.164 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-7.860 

.000 

6.974** 

GUARD 

1.193 

3.298 

5.377* 

GENDER 

.936 

2.549 

3.745* 

EL 

.089 

1.093 

8.868** 

*  =  p  <  .05  ***  =  p<.001 

**  =  p  <  .01 


A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  a  model,  including  the  EL  composite,  Army  component 
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(GUARD  =  National  Guard),  and  GENDER  (GENDER  =  Female),  accounts  for  about  16 
percent  of  the  variation  in  the  dependent  variable.  This  model  is  statistically  significant 
(X  =  25.393,  P  <  .001),  and  has  moderate  explanatory  power  (Nagelkerke  R  =  .164). 

There  is  a  statistically  significant  effect  for  the  EL  composite,  GUARD,  and 
GENDER.  The  model  suggests  that  having  a  higher  EL  composite  score  increases  the 
odds  of  passing.  At  the  mean  EL  score,  an  increase  of  one  point  is  associated  with  an 
increase  of  about  9%  in  the  odds  of  passing  the  course,  and  a  five-point  increase  in  EL 
would  increase  the  odds  of  passing  by  56%.  National  Guard  membership  increases  the 
odds  of  passing  by  230%.  Female  Soldiers  have  155%  higher  odds  of  passing  than  male 
Soldiers  do. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Regular  Army 
Soldier  from  the  larger  Army  contract  population  would  pass  the  course  based  upon  the 
binary  logistic  model.  These  were  calculated  using  the  formula  for  finding  probability  in 
a  binary  logistic  model,  which  is 

qw'x)  /  (1  +  q(P’x)). 

The  following  analysis  will  be  based  upon  Regular  Army  Soldiers.  Based  on  the  model, 
when  the  cutoff  score  is  at  its  current  level  (EL  =  95),  male  Soldiers  with  an  average  EL 
score  (EL  =  109.93)  have  approximately  an  87%  chance  of  passing,  while  female 
Soldiers  have  approximately  a  95%  chance  of  passing.  Currently,  about  77%  of  Soldiers 
are  eligible  for  MOS  27M  assignment  at  the  current  cutoff.  Lowering  the  cutoff  by  five 
points  (EL  =  90)  would  increase  eligibility  by  14  percentage  points  (to  90%)  and  the 
average  male  Soldier  who  would  qualify  for  training  would  have  a  slightly  lower  chance 
of  passing  (85%),  as  would  the  average  female  Soldier  (93%).  Raising  the  cutoff  score 
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by  five  points  (EL  =  100),  the  average  male  Soldier  would  have  a  slightly  higher  chance 
of  passing  (89%),  as  would  the  average  female  Soldier  (96%).  However,  eligibility  would 
fall  by  about  13  percentage  points. 


Table  2.  27M:  Probability  that  a  Regular  Army  Soldier  (from  the  larger  Army 
contract  population)  will  pass  the  course  based  on  the  binary  logistic  model 


Cutoff  = 

85 

Cutoff = 

90 

Cutoff = 

95 

Cutoff = 
100 

Cutoff = 
105 

Percent  Eligible 
(Regular  Army) 

96.9% 

89.8% 

78.5% 

65.9% 

51.1% 

Mean 

106.22 

107.71 

109.93 

112.38 

115.44 

Passing  Rate: 

Male 

83.1% 

84.9% 

87.3% 

89.5% 

91.8% 

Female 

92.6% 

93.5% 

94.6% 

95.6% 

96.6% 

Figure  1  shows  the  relationship  between  EL  and  the  probability  of  passing  for 
Regular  Army  Soldiers,  including  upper  and  lower  bounds  based  upon  the  standard  error 
of  the  estimated  EL  coefficient.  For  a  particular  EL  score,  trace  a  vertical  line  up  to  the 
curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the  corresponding 
probability  of  passing.  For  example,  an  EL  score  of  100  corresponds  to  a  passing 
probability  of  about  75%  for  a  male  Soldier  and  88%  for  a  female  soldier.  One  can  see 
that  high  school  educated  Soldiers  with  “low”  EL  scores  have  a  poor  chance  of  passing, 
but  the  chance  of  passing  increases  somewhat  quickly  as  scores  increase. 
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Figure  1:  Predicted  Probability  of  Passing:  Regular  Army  Soldiers 
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EL  Composite  Score 


Appendix:  Soldier  Characteristics  (27M) 


Gender 

Outcome 

Male 

Female 

19.0% 

10.9% 

81.0% 

89.1% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

.0% 

19.0% 

7.1% 

100.0% 

81 .0% 

92.9% 

Branch  of  Army 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

.0% 

6.4% 

pass 

.0% 

93.6% 
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27M  Risk  Analysis  of  Policy  Alternatives 


In  this  section  we  extend  the  policy  analysis  with  a  risk  analysis  of  the  policy 
alternatives  considered  using  simulation  methods.  We  employ  the  “@RISK”  software 
package  which  is  designed  as  an  add-in  for  Microsoft  Excel.66  There  are  four  steps  in  the 
risk  analysis. 

The  first  step  is  developing  a  model.  A  regression  model  for  predicting  the 
passing  rate  has  been  specified  and  estimated,  and  the  results  stored  in  Excel  worksheet 
format.  See  Table  1. 

The  second  step  is  identifying  the  uncertainty  inherent  in  both  the  input  variables 
(composite  scores,  Soldier  characteristics)  and  the  corresponding  estimated  parameters 
and  specifying  their  possible  values  with  probability  distributions. 

To  approximate  the  governing  composite  distribution,  we  use  the  larger  Army 
contract  population  (EAF  file:  those  who  contracted  from  Jan  1992  to  Aug  2003).  This 
distribution  can  be  assumed  to  follow  a  normal  distribution.  The  mean  and  standard 
deviation  estimated  from  the  contract  population  and  used  as  input  to  the  simulations  are 
shown  in  Table  R-l  (before  truncation). 


Table  R-l.  Input  infonnation  for  the  simulations 

Mean 

Std  error  / 
deviation 

Constant 

-7.860 

2.976 

EL  var 

105.33  (before  truncation) 

12.57 

EL  coeff 

.089 

.030 

GENDER  var 

22.5%(female) 

GENDER  coeff 

.936 

.484 

GUARD  var 

32% 

GUARD  coeff 

1.193 

.515 

_ 

To  approximate  the  distributions  for  the  demographic  variables  (GENDER,  and 
GUARD)  we  rely  on  the  training  data  sample  used  to  estimate  the  regression  model. 
These  distributions  can  be  assumed  to  follow  binomial  distributions  with  means 
estimated  from  the  training  data  sample  as  shown  in  Table  R-l. 

To  approximate  the  distributions  for  the  estimated  parameters  in  the  logit  model, 
we  note  that,  under  very  general  conditions,  the  estimates  will  be  asymptotically  normally 


66  Software  is  available  from  Palisade  Corporation,  Newfield,  NY. 
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distributed.  The  estimated  coefficients  (means  and  standard  errors)  are  shown  in  Table 
R-l. 


The  third  step  is  analyzing  the  model  with  simulation  to  determine  the  range  and 
probabilities  of  all  possible  training  outcome  results.  The  results  reported  represent  the 
simulation  of  the  regression  prediction  equation  for  1000  iterations  using  a  fixed  seed.68 
The  comparisons  are  between  the  baseline  case  (EL  =  95)  and  the  policy  cases  (EL  =  90 
and  EL  =  100).  These  cases  are  delineated  by  the  cutoff  score  level  which  serves  as  the 
lower  truncation  point  in  the  governing  composite  input  distribution.  The  simulation 
results  in  Table  R-2  describe  the  effect  on  the  mean  probability  of  passing  as  the  EL 
cutoff  score  level  is  varied,  and  represent  an  improvement  of  the  policy  analysis 
presented  in  Table  2  because  they  reflect  the  simultaneous  effects  of  all  known  sources  of 
uncertainty. 


Table  R-2.  27M  simulation  results 


EL  =  90 

EL  =  95 

o 

o 

II 

1-1 

W 

Mean  governing 
composite 

107.76 

109.53 

111.90 

Mean  probability 
of  passing 

73.9% 

75.0% 

76.1% 

Std  deviation 

33.3% 

32.8% 

32.3% 

The  frequency  distributions  of  simulated  effects  are  shown  below  in  the  histogram 
graphs.  The  histogram  portrays  the  full  range  of  outcomes  that  average  to  the  mean 
effect  as  reported.  Looking  at  the  histogram  for  the  baseline  case  (EL  =  95),  probability 
of  passing  is  plotted  on  the  horizontal  axis  and  the  height  of  each  bar  indicates  the 
relative  number  of  Soldiers  at  that  passing  probability.  The  information  in  the  histograms 
is  effectively  summarized  in  Table  R-3.  In  that  table  we  see  the  percentage  of  Soldiers 
that  meet  or  exceed  a  given  chance  of  passing.  For  example,  looking  at  the  baseline  case 
(EL  =  95),  about  69%  of  Soldiers  have  a  70%  or  greater  chance  of  passing. 


67  The  intuition  is  that  logit  estimates  are  derived  from  maximum  likelihood  (ML)  estimation,  and  under 
fairly  general  conditions  ML  estimates  are  consistent  and  asymptotically  normally  distributed. 

68  Latin  Hypercube  (LH)  sampling  is  utilized  in  drawing  samples  for  the  input  distributions.  It  is  designed 
to  accurately  recreate  the  input  distribution  through  sampling  in  fewer  iterations  when  compared  with  the 
Monte  Carlo  method.  The  key  to  LH  sampling  is  stratification  of  the  input  probability  distributions. 
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We  find  that  the  mean  probability  of  passing  in  the  simulation  is  75%  (Table  R-2, 
baseline).  Looking  at  the  histograms,  we  see  a  large  portion  of  Soldiers  in  the  highest 
range  of  passing  scores,  and  the  remainder  spread  out  over  the  entire  range.  Sensitivity 
analysis  indicates  that  we  can  attribute  this  pattern  to  the  relatively  large  standard  errors 
on  the  governing  composite  and  constant  terms  that  come  into  play  in  the  simulation. 

The  relatively  large  standard  errors  introduce  variability  into  the  passing  probabilities69. 
We  note  that  the  mean  simulated  passing  probability  turns  out  to  be  about  ten  percentage 
points  lower  than  the  static  prediction  at  the  mean  EL  score  (Table  2).  In  the  simulated 


Table  R-3.  Percent  of  simulated  Soldiers  that  meet  or 
exceed  given  chance  of  passing 


Chance  of 
passing 

EL  =  90 

EL  =  95 

o 

o 

II 

_l 

HI 

95.0% 

44.3% 

47.0% 

50.8% 

90.0% 

53.2% 

55.8% 

57.4% 

85.0% 

58.5% 

60.5% 

61.9% 

80.0% 

62.8% 

64.6% 

64.7% 

75.0% 

67.0% 

67.3% 

68.4% 

70.0% 

68.6% 

69.3% 

70.3% 

65.0% 

70.6% 

71.7% 

72.8% 

60.0% 

72.3% 

74.0% 

74.9% 

55.0% 

74.6% 

76.1% 

76.0% 

50.0% 

76.4% 

77.9% 

77.6% 

predictions  we  are  calculating  an  average  over  all  Soldiers  and  taking  into  account  all 
sources  of  uncertainty,  whereas  in  the  static  prediction  we  are  calculating  the  probability 
of  passing  only  at  the  mean  EL  score,  ignoring  the  distribution  of  Soldier  scores  etc.  as 
well  as  uncertainty.  Recognizing  this  variability  is  important,  but  we  also  note  that  the 
policy  case  impacts  relative  to  the  baseline  are  the  same  in  both  simulation  and  static 
prediction  modes. 

The  fourth  step  is  making  a  decision  based  on  the  results  provided  and  the 
preferences  of  the  school  proponent.  This  involves  the  evaluation  by  the  school 
proponent  of  the  tradeoff  between  increased  eligibility  and  reduced  passing  probabilities 
(and  vice-versa). 


69  The  large  standard  error  for  the  constant  allows  scores  to  be  more  spread  out  and  allows  for  large 
groupings  at  the  highest  and  lowest  values.  And  since  the  probabilities  of  passing  are  calculated  from  the 
ratio:  e^' x>  /  (1  +  XJ),  as  the  scores  calculated  from  the  equation  become  extreme,  the  ratios  are  more 
likely  to  approach  zero  and  one. 
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Distribution  for  PASS  EQUATION  95 


X  <=0.04  X  <=1 


Distribution  for  PASS  EQUATION  100 

X  <=0.06  X  <=1 
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Distribution  for  PASS  EQUATION  90 

X<=0.03  X<=1 
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96B:  Intelligence  Analyst 


The  final  sample  included  2863  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (78.4%)  or  failed  for  academic  reasons  (21.6%)  were 
included  in  the  analysis  sample  of  1621  Soldiers  because  we  assume  that  non-completion 
for  non-academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (76%,  24%  female),  most  had  a  high  school 
diploma  but  not  more  education  (79%,  18%  some  college,  3%  GED  or  less),  and  most 
were  from  were  from  the  Regular  Anny  (72%,  19%  National  Guard,  9%  Reserve).  The 
governing  AA  composite,  Skilled  Technical  (ST),  for  this  MOS  has  a  cutoff  score  of  102; 
the  sample  mean  is  1 16.53  (standard  deviation  =  8.844).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 


Table  1.  96B:  Results  of  the  binary  logistic 
prediction  model 

Chi-  Square 

295.374*** 

Log  Likelihood 

2688.809 

Nagelkerke  R  Square 

.151 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-9.433 

.000 

152.297*** 

GED 

.730 

2.075 

3.399 

COLLEGE 

.400 

1.491 

7.352** 

RESERVE 

.494 

1.639 

6.113* 

GUARD 

.624 

1.867 

19.595*** 

GENDER 

.374 

1.454 

10.724** 

ST 

.091 

1.095 

184.318*** 

*  =  p  <  .05  ***  =  p<.00] 

**  =  p  <  .01 

[ 
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Table  1  and  indicate  that  a  model,  including  the  ST  composite,  education  level  (GED  = 
GED  or  less  than  high  school  diploma;  COLLEGE  =  some  college),  Army  component 
(GUARD  =  National  Guard;  RESERVE  =  Army  Reserve)  and  gender  (GENDER  = 
female),  accounts  for  about  15%  of  the  variation  in  the  dependent  variable.  This  model  is 
statistically  significant  (%2  =  295.374,  P  <  .001),  and  has  moderate  explanatory  power 
(Nagelkerke  R2  =  .  15 1). 

There  is  a  statistically  significant  effect  for  the  ST  composite.  The  model 
suggests  that  having  a  higher  ST  composite  score  increases  the  odds  of  passing.  At  the 
mean  ST  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  10%  in 
the  odds  of  passing  the  course  (on  the  first  attempt),  and  a  five-point  increase  in  ST 
would  increase  the  odds  of  passing  by  58%.  There  are  also  statistically  significant  effects 
for  COLLEGE,  GENDER,  RESERVE,  and  GUARD,  with  increased  odds  of  passing  the 
course  at  49%,  45%  (for  females),  64%,  and  87%,  respectively. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

qw'x)  /  (1  +  q(P’x>). 

The  following  analysis  is  focused  upon  male  and  female  Soldiers  with  a  high  school 
diploma  or  some  college  education  from  the  Regular  Army.  When  the  cutoff  score  is  at 
its  current  level  (ST  =  102),  male  Soldiers  with  a  high  school  diploma  (the  modal 
categories)  and  an  average  ST  score  (ST  =  1 14.16)  have  approximately  a  72%  chance  of 
passing  (on  the  first  attempt).  Currently,  about  65%  of  Soldiers  are  eligible  for  MOS  96B 
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assignment  at  the  current  cutoff  (ST  =  102).  Lowering  the  cutoff  by  five  points  (ST  =  97) 
would  increase  eligibility  by  13  percentage  points  (to  78%),  while  the  average  male 
Soldier  who  would  qualify  for  training  would  have  a  somewhat  lower  chance  of  passing 
(67%).  Raising  the  cutoff  score  by  five  points  (ST  =  107),  the  average  male  Soldier 
would  have  a  higher  chance  of  passing  (77%)  but  eligibility  would  fall  by  about  15 
percentage  points. 


Table  2.  96B:  Probabi 
population)  will  pass  t 

lity  that  a  Regular  Army  Soldier  (from  the  larger  Anny  contract 
le  course  based  on  the  binary  logistic  model 

Cutoff  = 

92 

Cutoff = 

97 

Cutoff = 
102 

Cutoff = 
107 

Cutoff = 
112 

Percent  Eligible 
(Regular  Army) 

89.0% 

78.2% 

64.9%o 

49.6%, 

34.7% 

Mean 

109.45 

111.59 

114.16 

117.18 

120.51 

Passing  Rates: 

High  School  Male 

i maam: 

HB 

College  Male 

— — 

— 

High  School  Female 

■MM— 

College  Female 

Tmmm i 

K 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
male  Regular  Army  Soldiers  by  education,  including  upper  and  lower  bounds  based 
upon  the  standard  error  of  the  estimated  ST  coefficient.  For  a  particular  ST  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
corresponding  probability  of  passing.  For  example,  an  ST  score  of  100  corresponds  to  a 
passing  probability  of  about  40%  for  a  high  school  educated,  Regular  Army  male  Soldier. 
One  can  see  that  Soldiers  with  a  low  ST  score  stand  very  little  chance  of  passing  the 
course.  As  ST  scores  increase,  the  chance  that  a  Soldier  will  pass  the  course  increases 
sharply.  In  addition,  the  graph  illustrates  that  Soldiers  with  some  college  education  have  a 
noticeably  better  chance  of  passing  this  course. 
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Figure  1:  Predicted  Probalility  of  Passing:  Male  Soldiers 


|  High  School  -  -  -  HS  Upper  -  -  -  HS  Lower  ^^^College  —  -  Col  Upper  —  -  Col  Lower  | 

Figure  2  shows  the  relationship  between  ST  and  the  probability  of  passing  for 


high  school  educated  Soldiers  by  gender,  including  upper  and  lower  bounds  based  upon 
the  standard  error  of  the  estimated  ST  coefficient.  One  can  see  that  Soldiers  with  a  low 
ST  score  stand  very  little  chance  of  passing  the  course.  As  ST  scores  increase,  the  chance 
that  a  Soldier  will  pass  the  course  increases  sharply.  In  addition,  the  graph  illustrates  that 
female  Soldiers  have  a  noticeably  better  chance  of  passing  this  course. 
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Figure  2:  Predicted  Probalility  of  Passing:  High  School  Soldiers 


—  Male  -  -  -  Male  Upper  ■  -  -  Male  Lower  Female  —  ■  Female  Upper  —  ■  Female  Lower 


Figure  3  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
Soldiers  by  education  and  gender.  The  graph  illustrates  that  women  with  some  college 
education  have  the  best  chance  at  passing  the  course,  while  high  school  educated  males, 
the  modal  students,  have  the  lowest  chance  of  passing.  Males  with  some  college 
education  and  females  with  a  high  school  education  have  essentially  the  same  chances  of 
passing.  Figure  3  also  illustrates  the  magnitude  of  the  effect  brought  about  by  gender  and 
education.  At  the  current  cutoff  score  (ST  =  102)  college  educated  female  Soldiers  have 
more  than  a  15%  better  chance  of  passing  the  course  compared  to  high  school  educated 
males.  Even  at  a  ST  score  as  high  as  120,  college  educated  women  still  have 
approximately  a  10%  better  chance  of  passing  than  high  school  educated  males. 
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Chance  of  passing 


Figure  3:  HS  vs.  College  &  Male  vs.  Female 


ST  composite  score 


- M-HS  -  -  -  M-Col - F-HS  —  -  F-Col 
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Appendix:  Soldier  Characteristics  (96B) 


Gender 

Male 

Female 

Outcome 

fail 

21.0% 

23.4% 

pass 

79.0% 

76.6% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

pass 

9.9% 

90.1% 

24.0% 

76.0% 

12.8% 

87.2% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

pass 

24.6% 

75.4% 

13.2% 

86.8% 

14.1% 

85.9% 
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Appendix: 

A  note  on  the  relationship  between  academic  failure  and  non-academic  failure 

Example  -  96B 


Background 

In  previous  work,  we  estimated  a  binary  logistic  regression  model  to  predict  passing 
versus  failing.  The  analysis  was  restricted  to  so-called  academic  cases,  and  excluded 
failure  cases  for  non-academic  reasons.  The  explanatory  variables  were  the  ST 
composite,  education  status  (i.e.,  some  college  and  GED  or  less  education),  Anny 
component  (i.e.,  Army  Reserve  or  National  Guard),  and  Soldier’s  gender.  The  estimation 
results  are  reported  in  Table  1  and  indicate  that  this  model  accounts  for  about  15%  of  the 
variation  in  passing  vs.  failing.  This  model  is  statistically  significant  (x  =  295.374,  p  < 
.001),  and  has  moderate  explanatory  power  (Nagelkerke  R2  =  .151). 


We  now  proceed  to  examine  the  implicit  assumption  made  by  excluding  non- 
academic  failure  cases  -  i.e.,  that  non-academic  failure  cases  are  fundamentally  different 
from  academic  failure  cases. 


Table  1.  96B:  Results  of  the  binary  logistic 
prediction  model 

Chi-  Square 

295.374*** 

Log  Likelihood 

2688.809 

Nagelkerke  R  Square 

.151 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-9.433 

.000 

152.297*** 

GED 

.730 

2.075 

3.399 

COLLEGE 

.400 

1.491 

7.352** 

RESERVE 

.494 

1.639 

6.113* 

GUARD 

.624 

1.867 

19.595*** 

GENDER 

.374 

1.454 

10.724** 

ST 

.091 

1.095 

184.318*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p<  .01 

[ 
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Multinomial  model 


A  multinomial  logistic  regression  was  estimated  in  order  to  examine  the  relationship 
between  academic  failure,  non-academic  failure,  and  passing.  Academic  failure  was  used 
as  the  reference  category,  which  means  that  non-academic  failure  and  passing  are 
contrasted  to  it  and  not  to  each  other.  A  model  including  the  ST  composite  score, 
education  level  (i.e.,  GED  and  COLLEGE),  gender,  and  Army  component  (GUARD  and 
RESERVE)  significantly  fit  the  data,  %2  (12)  =  316.724,  p  <  .001.  The  Nagelkerke  R2 
equals  .  1 1 0  as  compared  to  a  Nagelkerke  R  of .  1 5 1  from  the  binary  logistic  model.  This 
difference  of  four  percentage  points  suggests  that  adding  the  non-academic  failure 
category  produces  a  weaker  overall  model.  In  other  words,  the  variables  included  in  the 
model  predict  passing  better  than  they  predict  both  passing  and  non-academic  failure. 

Table  2  shows  summary  statistics  for  the  multinomial  model.  When  the 
significance  of  the  Chi-square  statistic  for  the  variables  is  compared  to  the  significance  of 
the  Wald  statistic  in  Table  1,  it  is  notable  that  the  same  variables  are  significant.  This 
demonstrates  that  adding  a  third  category  to  the  dependent  variable  in  the  logistic 
regression  does  not  affect  the  overall  utility  of  the  logistic  comparison  between  academic 
failure  and  passing. 


Table  2.  Multinomial  model:  summary 
statistics 


Chi-square 

316.734*** 

Nagel 

kerke  R  Square 

.110 

Variable 

-2  Log 
Likelihood 

r 

Intercept 

1588.391 

.000 

GED 

1793.480 

5.834 

COLLEGE 

1602.950 

14.559*** 

RESERVE 

1597.032 

8.641* 

GUARD 

1611.560 

23.169*** 

GENDER 

1621.566 

33.175*** 

ST 

1793.480 

205.088*** 

*  p  < .05  ***p  < .001 

**  p  <  .01 


Table  3  presents  the  coefficients  for  the  model’s  prediction  of  passing  vs. 
academic  failure  as  estimated  in  the  multinomial  logistic  model.  The  results  are 
essentially  the  same  as  those  from  the  binary  logistic  model  reported  in  Table  1. 7 


70  SPSS  allows  the  user  to  assign  the  reference  group  for  binary  logistic  regression.  Because  of  this,  the 
demographic  variables  were  coded  as  dummy  variables  with  the  modal  categories  (i.e.,  more  than  a  GED; 
less  than  some  college  education;  not  in  the  Army  Reserve;  not  in  the  National  Guard;  and  male)  coded  as 
zero.  In  contrast,  the  multinomial  logistic  function  in  SPSS  automatically  sets  the  highest  category  of  an 
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Table  3.  Multinomial  model:  passing  versus  academic 
failure 


Variable 

Coefficient 

Odds  Ratio 

Wald 

Intercept 

-8.945 

NA 

143.922*** 

GED 

.710 

2.034 

3.248 

COLLEGE 

.444 

1.554 

9.011** 

RESERVE 

.554 

1.740 

7.847** 

GUARD 

.638 

1.893 

20.481*** 

GENDER 

.348 

1.416 

9.472** 

ST 

.086 

1.090 

176.081*** 

*  p  < .05  ***p  < .001 

**  p  <  .01 


Table  4  presents  the  coefficients  for  the  model’s  prediction  of  non-academic 
failure  vs.  academic  failure  (the  reference  category).  The  model  for  predicting  non- 
academic  failure  closely  resembles  that  for  predicting  passing.  This  implies  several 
things.  First,  non-academic  failures  are  different  from  academic  failures;  if  they  were 
similar,  this  model  would  be  expected  to  have  many  fewer  significant  predictors. 
However,  the  only  changes  are  new  significance  for  GED  and  lack  of  significance  for 
RESERVE.  Most  importantly  the  ST  composite  is  still  significant,  where  a  higher  ST 
score  predicts  non-academic  failure!  This  suggests  that  when  Soldiers  with  high  ST 
scores  leave  the  school,  it  is  not  for  cognitive  reasons.  The  fact  that  the  model  does 
resemble  the  model  for  graduates  so  closely  suggests  that  non-academic  failure  is 
fundamentally  different  from  academic  failure. 


Table  4.  Multinomial  model:  non-academic  versus 
academic  failure 


Variable 

Coefficient 

Odds  Ratio 

Wald 

Intercept 

-8.893 

N/A 

88.371*** 

GED 

1.031 

2.804 

5.529* 

COLLEGE 

.646 

1.907 

13.549*** 

RESERVE 

.456 

1.578 

3.551 

GUARD 

.398 

1.489 

4.191* 

GENDER 

.812 

2.251 

32.760*** 

ST 

.072 

1.075 

77.287*** 

*  p  <  .05  ***p  <  .001 

**p<  .01 


independent  dichotomous  variable  to  be  the  reference  category.  Thus,  we  recoded  the  variables  so  that  the 
modal  categories  equal  one. 
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96B  Risk  Analysis  of  Policy  Alternatives 


In  this  section  we  extend  the  policy  analysis  with  a  risk  analysis  of  the  policy 
alternatives  considered  using  simulation  methods.  We  employ  the  “@RISK”  software 
package  which  is  designed  as  an  add-in  for  Microsoft  Excel.71  There  are  four  steps  in  the 
risk  analysis. 

The  first  step  is  developing  a  model.  A  regression  model  for  predicting  the 
passing  rate  has  been  specified  and  estimated,  and  the  results  stored  in  Excel  worksheet 
format.  See  Table  1. 

The  second  step  is  identifying  the  uncertainty  inherent  in  both  the  input  variables 
(composite  scores,  Soldier  characteristics)  and  the  corresponding  estimated  parameters 
and  specifying  their  possible  values  with  probability  distributions. 

To  approximate  the  governing  composite  distribution,  we  use  the  larger  Army 
contract  population  (EAF  file:  those  who  contracted  from  Jan  1992  to  Aug  2003).  This 
distribution  can  be  assumed  to  follow  a  normal  distribution.  The  mean  and  standard 
deviation  estimated  from  the  contract  population  and  used  as  input  to  the  simulations  are 
shown  in  Table  R-l  (before  truncation). 


Table  R-l.  Input  infonnation  for  the  simulations 

Mean 

Std  error  / 
deviation 

Constant 

-9.433 

.764 

ST  var 

107.02  (before  truncation) 

12.157 

ST  coeff 

.091 

.007 

GED  var 

2.8% 

GED  coeff 

.730 

.396 

COLLEGE  var 

18.2% 

COLLEGE  coeff 

.400 

.147 

GUARD  var 

19.3% 

GUARD  coeff 

.624 

.141 

RESERVE  var 

9.0% 

RESERVE  coeff 

.494 

.200 

GENDER  var 

24.2% 

GENDER  coeff 

.374 

.114 

71  Software  is  available  from  Palisade  Corporation,  Newfield,  NY. 
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To  approximate  the  distributions  for  the  demographic  variables  we  rely  on  the 
training  data  sample  used  to  estimate  the  regression  model.  These  distributions  can  be 
assumed  to  follow  binomial  distributions  with  means  estimated  from  the  training  data 
sample  as  shown  in  Table  R-l. 

To  approximate  the  distributions  for  the  estimated  parameters  in  the  logit  model, 
we  note  that,  under  very  general  conditions,  the  estimates  will  be  asymptotically  normally 
distributed.72  73  The  estimated  coefficients  (means  and  standard  errors)  are  shown  in 
Table  R-l. 

The  third  step  is  analyzing  the  model  with  simulation  to  determine  the  range  and 
probabilities  of  all  possible  training  outcome  results.  The  results  reported  represent  the 
simulation  of  the  regression  prediction  equation  for  1000  iterations  using  a  fixed  seed.74 
The  comparisons  are  between  the  baseline  case  (ST  =  102)  and  the  policy  cases  (ST  =  97 
and  ST  =  107).  These  cases  are  delineated  by  the  cutoff  score  level  which  serves  as  the 
lower  truncation  point  in  the  governing  composite  input  distribution.  The  simulation 
results  in  Table  R-2  describe  the  effect  on  the  mean  probability  of  passing  as  the  ST 
cutoff  score  level  is  varied,  and  represent  an  improvement  of  the  policy  analysis 
presented  in  Table  2  because  they  reflect  the  simultaneous  effects  of  all  known  sources  of 
uncertainty. 


Table  R-2.  96B  simulation  resul 

Its 

C/5 

H 

II 

"'4 

ST  =  102 

ST  =  107 

Mean  governing 
composite 

111.36 

113.76 

116.71 

Mean  probability 
of  passing 

67.81% 

71.68% 

76.01% 

Std  deviation 

23.65% 

21.69% 

19.99% 

The  frequency  distributions  of  simulated  effects  are  shown  below  in  the  histogram 
graphs.  The  histogram  portrays  the  full  range  of  outcomes  that  average  to  the  mean 
effect  as  reported.  Looking  at  the  histogram  for  the  baseline  case  (ST  =  102),  probability 


72  The  intuition  is  that  logit  estimates  are  derived  from  maximum  likelihood  (ML)  estimation,  and  under 
fairly  general  conditions  ML  estimates  are  consistent  and  asymptotically  normally  distributed. 

73  An  alternative  to  relying  on  asymptotics  is  to  utilize  a  bootstrap  procedure  to  estimate  the  distribution  of 
the  logit  estimates.  We  found  the  bootstrap  estimates  for  the  standard  error  of  the  governing  composite 
coefficient  to  be  virtually  identical  with  the  original  estimate.  We  did  find  small  differences  for  the  other 
standard  error  estimates  vis-a-vis  the  original  estimates  but  the  simulation  results  with  the  revised  standard 
errors  were  unaffected. 

74  Latin  Hypercube  (LH)  sampling  is  utilized  in  drawing  samples  for  the  input  distributions.  It  is  designed 
to  accurately  recreate  the  input  distribution  through  sampling  in  fewer  iterations  when  compared  with  the 
Monte  Carlo  method.  The  key  to  LH  sampling  is  stratification  of  the  input  probability  distributions. 
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of  passing  is  plotted  on  the  horizontal  axis  and  the  height  of  each  bar  indicates  the 
relative  number  of  Soldiers  at  that  passing  probability.  The  information  in  the  histograms 
is  effectively  summarized  in  Table  R-3.  In  that  table  we  see  the  percentage  of  Soldiers 
who  meet  or  exceed  a  given  chance  of  passing.  For  example,  looking  at  the  baseline  case 
(ST  =  102),  about  61%  of  Soldiers  have  a  70%  or  greater  chance  of  passing. 


Table  R-3. 

Percent  of  simulated  Soldiers  that  meet  or  exceed 
given  chance  of  passing 


Chance  of 
passing 

ST  =  97 

ST  =  102 

ST  =  107 

95.0% 

9.3% 

12.6% 

14.7% 

90.0% 

21.3% 

22.7% 

30.4% 

85.0% 

30.6% 

34.1% 

42.5% 

80.0% 

39.2% 

44.6% 

52.9% 

75.0% 

47.2% 

53.8% 

62.1% 

70.0% 

54.0% 

60.9% 

70.0% 

65.0% 

60.1% 

68.1% 

75.2% 

60.0% 

65.9% 

73.1% 

79.8% 

55.0% 

70.8% 

77.8% 

84.6% 

50.0% 

76.2% 

82.1% 

88.4% 

We  find  that  the  mean  probability  of  passing  in  the  simulation  is  72%  (Table  R-2, 
baseline).  Looking  at  the  histograms,  we  see  the  largest  portion  of  Soldiers  is  in  the 
highest  range  of  passing  scores,  and  with  a  consistent  decline  as  the  probability  of 
passing  approaches  zero.  We  note  that  the  mean  simulated  passing  probability  turns  out 
to  be  only  slightly  lower  than  the  static  prediction  at  the  mean  ST  score  (Table  2).  In  the 
simulated  predictions  we  are  calculating  an  average  over  all  Soldiers  and  taking  into 
account  all  sources  of  uncertainty,  whereas  in  the  static  prediction  we  are  calculating  the 
probability  of  passing  only  at  the  mean  ST  score,  ignoring  the  distribution  of  Soldier 
scores  etc.  as  well  as  uncertainty.  While  recognizing  this  variability  is  important,  we  also 
note  that  the  policy  case  impacts  relative  to  the  baseline  are  the  same  in  both  simulation 
and  static  prediction  modes. 

The  fourth  step  is  making  a  decision  based  on  the  results  provided  and  the 
preferences  of  the  school  proponent.  This  involves  the  evaluation  by  the  school 
proponent  of  the  tradeoff  between  increased  eligibility  and  reduced  passing  probabilities 
(and  vice-versa). 
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Distribution  for  PASS  EQUATION  102 

X  <=0.29 


Distribution  for  PASS  EQUATION  107 
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Distribution  for  PASS  EQUATION  97 


X  <=0.25  X  <=0.97 

100% 

90% 

80% 
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60% 

50% 

40% 

30% 

20% 

10% 

0% 

0  0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9  1 


5%  95% 
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96D:  Imagery  Analyst 


The  final  sample  included  529  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  for  this  MOS  during  a  period  from  2001 
to  2004.  Only  Soldiers  who  graduated  (86.2%)  or  failed  for  academic  reasons  (13.8%) 
were  included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (69%,  31%  female),  most  had  a  high  school 
diploma  but  not  more  education  (80%,  20%  some  college),  and  most  were  from  were 
from  the  Regular  Army  (85%,  5%  National  Guard,  10%  Reserve).  The  governing  AA 
composite,  Skilled  Technical  (ST),  for  this  MOS  has  cutoff  score  of  102;  the  sample 
mean  is  1 14.66  (standard  deviation  =  8.366).  See  the  Appendix  for  a  description  of 
Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  model,  including  the  ST  composite,  education  status 
(COLLEGE  =  some  college  education),  Army  component  (GUARD  =  National  Guard, 
RESERVE  =  Army  Reserve),  and  gender  (GENDER  =  Female)  accounts  for  about  nine 

percent  of  the  variation  in  the  dependent  variable.  This  model  is  statistically  significant 

2  2 

(X  =  26.633,  P  <  .001),  and  has  moderate  explanatory  power  (Nagelkerke  R  =  .089). 
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Table  1.  96D:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

26.633*** 

Log  Likelihood 

397.952 

Nagelkerke  R  Square 

.089 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-7.252 

.001 

10.626*** 

COLLEGE 

.221 

1.247 

.310 

GUARD 

-.127 

.881 

.048 

RESERVE 

.193 

1.213 

.148 

GENDER 

-.176 

.839 

.409 

ST 

.081 

1.084 

16.510*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =p  <  .01 


There  is  a  statistically  significant  effect  for  the  ST  composite.  The  model 
suggests  that  having  a  higher  ST  composite  score  increases  the  odds  of  passing.  At  the 
mean  ST  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  eight 
percent  in  the  odds  of  passing  the  course,  and  a  four-point  increase  in  ST  would  increase 
the  odds  of  passing  by  50%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  calculations  were  made  using  the  formula  for  finding  probability  in  a 
binary  logistic  model,  which  is 

Q(P’x)  /  (1  + 

Because  COLLEGE,  GUARD,  RESERVE,  and  GENDER  were  not  statistically 
significant,  this  analysis  is  confined  to  the  modal — high  school  educated  male  Soldiers 
from  the  Regular  Army  (though  results  would  be  similar  for  any  demographic 
combination).  Based  on  the  model,  when  the  cutoff  score  is  at  its  current  level  (ST  = 
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102),  male  Soldiers  with  a  high  school  diploma  and  an  average  ST  score  (ST  =  1 14.66) 
have  approximately  an  88%  chance  of  passing.  Currently,  about  65%  of  Soldiers  are 
eligible  for  MOS  96D  assignment  at  the  current  cutoff.  Lowering  the  cutoff  by  five 
points  (ST  =  97)  would  increase  eligibility  by  seven  percentage  points  (to  13%)  and  the 
average  male  Soldier  who  would  qualify  for  training  would  have  a  slightly  lower  chance 
of  passing  (85%).  Raising  the  ST  cutoff  score  by  five  points  (ST  =  107)  provides  slightly 
higher  passing  rates  for  the  average  high  school  male  Soldier  (90%),  but  fewer  Soldiers 
would  be  eligible  (50%). 


Table  2.  96D:  Probability  that  a  Regular  Army  Soldier  (from  the  larger  successful 
applicant  population)  will  pass  the  course  based  on  the  binary  logistic  model _ 


Cutoff: 

ST  =  92 

Cutoff: 

ST  =  97 

Cutoff 

ST  =  102 

Cutoff 

ST  =  107 

Cutoff 

ST  =  112 

Percent  Eligible 
(Regular  Army) 

89.0% 

78.2% 

64.9%o 

49.6% 

34.7%, 

Mean 

109.06 

111.14 

113.60 

116.55 

119.83 

Passing  Rates: 

High  School  Male 

82.9% 

85.2% 

87.5% 

89.9% 

92.1% 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
high  school  educated  male  Regular  Army  Soldiers.  For  a  particular  ST  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
corresponding  probability  of  passing.  For  example,  a  ST  score  of  100  corresponds  to  a 
passing  probability  of  about  70%  for  a  high  school  educated,  Regular  Army  male  Soldier. 
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Chance  of  Passing 


Figure  1:  Predicted  Probability  of  Passing:  High  School  Soldiers 


ST  Composite  Scores 


-  -  -  |\/ia|e  Upper  -  -  -  Male  Lower 
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Appendix:  Soldier  Characteristics  (96D) 


Gender 

Outcome 

Male 

Female 

fail 

1 1 .8% 

18.2% 

pass 

88.2% 

81.8% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

14.8% 

8.5% 

pass 

85.2% 

91.5% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

14.2% 

9.6% 

15.4% 

pass 

85.8% 

90.4% 

84.6% 
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96H:  Common  Ground  Station  Operator 


The  final  sample  included  52 1  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  for  this  MOS  during  a  period  from  2001 
to  2004.  Only  Soldiers  who  graduated  (93.1%)  or  failed  for  academic  reasons  (6.9%) 
were  included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (80%,  20%  female),  most  had  a  high  school 
diploma  but  not  more  education  (84%,  15%  some  college,  1%  GED  or  less),  and  most 
were  from  were  from  the  Regular  Anny  (94%,  6%  National  Guard).  See  the  Appendix 
for  a  description  of  Soldier  characteristics  for  this  MOS  training  sample.  The  governing 
AA  composites,  Skilled  Technical  (ST)  and  Surveillance/Communications  (SC),  for  this 
MOS  both  have  cutoff  scores  of  102  and  93  respectively;  the  sample  mean  is  115.16 
(standard  deviation  =  8.501)  for  ST  and  1 12.75  (standard  deviation  =  8.971)  for  SC.  ST 
and  SC  are  correlated  at  the  .867  level.  See  the  Appendix  for  a  description  of  Soldier 
characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  model,  including  the  ST  composite,  education  status 
(COLLEGE  =  some  college  education)  and  gender  (GENDER  =  Female)  accounts  for 
about  6%  percent  of  the  variation  in  the  dependent  variable.  This  model  is  statistically 
significant  (x  =  12.457,  P  =  .006),  and  has  weak  explanatory  power  (Nagelkerke  R  = 
.060)75. 

75  A  model  was  estimated  using  only  the  ST  composite  due  to  the  dominance  of  the  ST  cutoff  over  the  SC 
cutoff  in  determining  eligible  Soldiers  for  assignment.  In  the  Appendix  we  report  models  estimated  with 
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Table  1.  96H:  Results  of  the  binary  logistic 
prediction  model  including  ST _ 


Chi-  Square 

12.457** 

Log  Likelihood 

249.396 

Nagelkerke  R  Square 

.060 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-5.197 

.006 

3.937* 

COLLEGE 

-.871 

.419 

4.323* 

GENDER 

.458 

1.580 

1.011 

ST 

.069 

1.072 

8.885** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =p  <  .01 


There  is  a  statistically  significant  effect  for  the  ST  composite.  The  model 
suggests  that  having  a  higher  ST  composite  score  increases  the  odds  of  passing.  At  the 
mean  ST  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  seven 
percent  in  the  odds  of  passing  the  course,  and  a  five-point  increase  in  SC  would  increase 
the  odds  of  passing  by  41%.  There  is  also  a  significant  effect  for  COLLEGE  where  those 
with  some  college  education  have  42%  lower  odds  of  passing. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

qw'x)  /  (1  +  q(P’x>). 

Because  component  and  gender  were  not  statistically  significant,  this  analysis  is  confined 

to  the  modal — high  school  (college)  educated  male  Regular  Army  Soldiers  (though 

results  would  be  similar  for  any  demographic  combination).  Based  on  the  model,  when 

both  SC  and  ST  composites,  and  with  SC  by  itself.  These  models  fit  noticeably  better  than  the  ST-only 
model. 
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the  cutoff  scores  are  at  its  current  level  (SC  =  93,  ST  =  102),  male  Soldiers  with  high 
school  and  an  average  ST  score  (ST  =  1 13.68)  have  approximately  a  93%  chance  of 
passing.  Currently,  about  63%  of  Soldiers  are  eligible  for  MOS  96H  assignment  at  the 
current  cutoff.  Lowering  the  cutoffs  by  five  points  (SC  =  88,  ST  =  102)  would  increase 
eligibility  by  nine  percentage  points  (to  77%)  and  the  average  male  Soldier  who  would 
qualify  for  training  would  have  a  about  the  same  chance  of  passing  at  92%.  Raising  the 
cutoff  scores  by  five  points  (SC  =  98,  ST  =  107)  provides  slightly  higher  passing  rates  for 


Table  2.  96H:  Probability  that  a  Regular  Army  Soldier  (from  the  larger  successful 
applicant  population)  will  pass  the  course  based  on  the  binary  logistic  model _ 


Cutoff: 

Cutoff: 

Cutoff: 

Cutoff: 

Cutoff: 

SC  =  83 

SC  =  88 

SC  =  93 

SC  =  98 

SC  =  103 

ST  =  92 

ST  =  97 

ST  =  102 

ST  =  107 

ST  =  112 

Percent  Eligible 
(Regular  Army) 

85.3% 

77.4% 

63.0% 

48.1% 

34. 6%> 

Mean 

109.04 

111.15 

113.68 

116.64 

119.96 

Passing  Rates: 

High  School  Male 

91.1% 

92.2% 

93.4% 

94.5% 

95.6% 

College  Male 

81.1% 

83.2% 

85.5% 

87.9% 

90.1% 

the  average  male  Soldier  at  95%,  but  fewer  Soldiers  would  be  eligible  (48%). 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
high  school  educated  male  Regular  Army  Soldiers.  For  a  particular  ST  score,  trace  a 
vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the 
corresponding  probability  of  passing.  For  example,  an  ST  score  of  100  corresponds  to  a 
passing  probability  of  about  85%  for  a  high  school  educated,  Regular  Army  male  Soldier. 
College  educated  male  Soldiers  consistently  have  a  lower  chance  of  passing  than  high 
school  educated  male  Soldiers 
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Chance  of  Passing 


Figure  1 :  Predicted  Probability  of  Passing:  High  School  Soldiers 


school 

-  -  -  High  school  Upper 

-  -  -  High  school  Lower 

College 

—  -  College  Upper 

—  -  College  Lower 
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Appendix:  Additional  Regression  Models 


Table  1.  96H:  Results  of  the  binary  logistic 
prediction  model  including  SC _ 


Chi-  Square 

26.500*** 

Log  Likelihood 

235.354 

Nagelkerke  R  Square 

.126 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-9.394 

.000 

12.349*** 

COLLEGE 

-.845 

.429 

3.860* 

GENDER 

.818 

2.267 

3.053 

SC 

.109 

1.115 

19.629*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 


Table  1.  96H:  Results  of  the  binary  logistic 
prediction  model  including  ST  and  SC 


Chi-  Square 

28.808*** 

Log  Likelihood 

233.046 

Nagelkerke  R  Square 

.136 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-7.686 

.000 

7.325*** 

COLLEGE 

-.797 

.451 

3.350 

GENDER 

.884 

2.420 

3.460 

SC 

.153 

1.165 

16.610*** 

ST 

-.058 

.944 

2.384 

*  =  p  <  .05  ***  =  p<.001 

**  =  p  <  .01 
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Appendix:  Soldier  Characteristics  (96H) 


Gender 

Outcome 

Male 

Female 

fail 

7.0% 

6.5% 

pass 

93.0% 

93.5% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

0.0% 

6.2% 

1 1 .4% 

pass 

100.0% 

93.8% 

88.6% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

7.2% 

0.0% 

2.9% 

pass 

92.8% 

0.0% 

97.1% 
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96U:  Unmanned  Aerial  Vehicle  Operator 


The  final  sample  included  1 16  soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  soldiers  who  graduated  (74.4%)  or  failed  for  academic  reasons  (26.6%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (92%,  9%  female),  most  had  a  high  school 
diploma  but  not  more  education  (91%,  7%  some  college,  2%  GED  or  less),  and  all  were 
Regular  Anny.  The  governing  AA  composite,  Surveillance/Communications(SC),  for 
this  MOS  has  a  cutoff  score  of  105;  the  sample  mean  is  121.84  (standard  deviation  = 
6.262).  See  the  Appendix  for  a  description  of  Soldier  characteristics  for  this  MOS 
training  sample. 


Table  1.  96U:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

25.1 19*** 

Log  Likelihood 

107.495 

Nagelkerke  R  Square 

.286 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-20.658 

.000 

14.728*** 

COLLEGE 

-1.418 

.242 

2.318 

GENDER 

.136 

1.146 

.032 

SC 

.195 

1.216 

15.871*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 


A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  model,  including  the  SC  composite,  education  status 
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(COLLEGE  =  some  college),  and  gender  (GENDER  =  female),  accounts  for  about  29% 
of  the  variation  in  the  dependent  variable.  This  model  is  statistically  significant  (y  = 

25.1 19,  P  <  .001),  with  moderate  explanatory  power  (Nagelkerke  R  =  .286). 

There  is  a  statistically  significant  effect  for  the  SC  composite.  The  model 
suggests  that  having  a  higher  SC  composite  score  increases  the  odds  of  passing.  At  the 
mean  SC  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  22%  in 
the  odds  of  passing  the  course,  and  a  five-point  increase  in  SC  would  increase  the  odds  of 
passing  by  165%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  fonnula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

Q(P'X>  /  (1  +  Q(fl’x)y 

Because  GENDER  was  not  statistically  significant,  this  analysis  is  confined  to  the  modal 
demographics — high  school  educated  male  Soldiers  from  the  Regular  Anny  (though 
results  would  be  similar  for  any  demographic  combination).  Based  on  the  model,  when 
the  cutoff  score  is  at  its  current  level  (SC  =  105),  Soldiers  with  an  average  SC  score  (SC 
=  120.5 1)  have  approximately  an  87%  chance  of  passing.  Currently,  about  only  51%  of 
soldiers  are  eligible  for  MOS  96U  assignment  at  the  current  cutoff  (SC  =  105).  Lowering 
the  cutoff  by  five  points  (SC  =  100)  would  increase  eligibility  by  13  percentage  points, 
while  the  average  Soldier  who  would  qualify  for  training  would  have  a  substantially 
lower  chance  of  passing  (79%).  Raising  the  cutoff  score  by  five  points  (SC  =  110),  the 
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average  Soldier  who  would  still  qualify  for  the  MOS  would  have  a  higher  chance  of 
passing  (92%),  but  considerably  fewer  Soldiers  would  be  eligible  (37%). 


Table  2.  96U:  Probability  that  a  Soldier  (from  the  larger  Army  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model _ _ _ 


Cutoff  = 

Cutoff = 

Cutoff = 

Cutoff = 

Cutoff = 

95 

100 

105 

110 

115 

Percent  Eligible 
( Regular  Army) 

77.0% 

63.7% 

50.8% 

36.8% 

25.1% 

Mean 

110.10 

112.80 

115.52 

118.74 

121.85 

Passing  rates: 

High  school  male 

69.2% 

79.2% 

86.6% 

92.4% 

95.7% 

Figure  1  shows  the  relationship  between  SC  and  the  probability  of  passing  for 
Regular  Army  Soldiers  with  high  school  education,  including  upper  and  lower  bounds 
based  upon  the  standard  error  of  the  estimated  SC  coefficient.  For  a  particular  SC  score, 
trace  a  vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to 
determine  the  corresponding  probability  of  passing.  For  example,  a  SC  value  of  100 
corresponds  to  a  passing  probability  of  about  25%  for  a  high  school  educated  male 
Soldier.  One  can  see  that  the  probability  of  passing  increases  steeply  over  the  full  range 
of  the  data. 
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Chance  of  Passing 


Figure  1 :  Predicted  Probalility  of  Passing:  Male  Soldiers  With  a  High  School  Diploma 


High  School  ■  -  -  HS  Upper  -  -  ■  HS  Lower 
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Appendix:  Soldier  Characteristics  (96U) 


Gender 

Outcome 

Male 

Female 

fail 

25.2% 

30.0% 

pass 

74.8% 

70.0% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

100.0% 

23.6% 

37.5% 

pass 

.0% 

76.4% 

62.5% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

25.6% 

.0% 

.0% 

pass 

74.4% 

.0% 

.0% 
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97B:  Counterintelligence  Analyst 


The  final  sample  included  1 162  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  for  this  MOS  during  a  period  from  2001 
to  2004.  Only  Soldiers  who  graduated  (90.8%)  or  failed  for  academic  reasons  (9.2%) 
were  included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (74%,  26%  female),  most  had  a  high  school 
diploma  but  not  more  education  (61%,  37%  some  college,  2%  GED  or  less),  and  most 
were  from  were  from  the  Regular  Anny  (67%,  18%  National  Guard,  15%  Reserve).  The 
governing  AA  composites,  Skilled  Technical  (ST),  for  this  MOS  has  cutoff  score  of  102; 
the  sample  mean  is  1 19.05  (standard  deviation  =  8.93 1).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  model,  including  the  ST  composite,  education  status  (GED  = 
GED  or  less  than  a  high  school  diploma,  COLLEGE  =  some  college  education),  Army 
component  (GUARD  =  National  Guard,  RESERVE  =  Army  Reserve),  and  gender 
(GENDER  =  Female)  accounts  for  about  21%  of  the  variation  in  the  dependent  variable. 

This  model  is  statistically  significant  (x  =  1 17.832,  P  <  .001),  and  has  moderate 

2 

explanatory  power  (Nagelkerke  R“  =  .210). 
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Table  1.  97B:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

1 17.832*** 

Log  Likelihood 

596.403 

Nagelkerke  R  Square 

.210 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-14.072 

.000 

61.306*** 

GED 

.430 

1.538 

.370 

COLLEGE 

.827 

2.286 

9.510** 

GUARD 

-.249 

.780 

.779 

RESERVE 

.226 

1.254 

.456 

GENDER 

.973 

2.645 

13.511*** 

ST 

.138 

1.148 

76.208*** 

*  =  p  <  .05  ***=p<.001 

**  =  p<  .01 


There  is  a  statistically  significant  effect  for  the  ST  composite.  The  model 
suggests  that  having  a  higher  ST  composite  score  increases  the  odds  of  passing.  At  the 
mean  ST  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  15%  in 
the  odds  of  passing  the  course,  and  a  five-point  increase  in  ST  would  increase  the  odds  of 
passing  by  99%.  There  are  also  statistically  significant  effects  for  COLLEGE  and 
GENDER,  with  increased  odds  of  passing  the  course  at  129%  (some  college)  and  165% 
(females),  respectively. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

Q(P'x)  /  (1  + 

The  following  analysis  is  focused  upon  male  and  female  Soldiers  with  a  high  school 
diploma  or  some  college  education  from  the  Regular  Army.  Based  on  the  model,  when 
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the  cutoff  score  is  at  its  current  level  (ST  =  102),  male  (female)  Soldiers  with  a  high 
school  diploma  and  an  average  ST  score  (ST  =  1 19.05)  have  approximately  an  84% 
(93%)  chance  of  passing.  Currently,  about  65%  of  Soldiers  are  eligible  for  MOS  97B 
assignment  at  the  current  cutoff.  Lowering  the  cutoff  by  five  points  (ST  =  97)  would 
increase  eligibility  by  14  percentage  points  (to  79%)  and  the  average  male  (female) 
Soldier  who  would  qualify  for  training  would  have  a  lower  chance  of  passing  (79%) 
(91%).  Raising  the  ST  cutoff  score  by  five  points  (ST  =  107)  leads  to  an  increase  in 
passing  rates  for  the  average  high  school  male  (female)  Soldier  (89%)  (96%),  but  fewer 
Soldiers  would  be  eligible  (50%). 


Table  2.  97B:  Probability  that  a  Regular  Anny  Soldier  (from  the  larger  successful 
applicant  population)  will  pass  the  course  based  on  the  binary  logistic  model 

Cutoff: 

ST  =  92 

Cutoff: 

ST  =  97 

Cutoff 

ST  =  102 

Cutoff 

ST  =  107 

Cutoff 

ST  =  112 

Percent  Eligible 
(Regular  Army) 

89.3% 

78.8% 

64.8% 

49. 6%> 

36.2% 

Mean 

109.34 

111.36 

114.05 

117.08 

120.08 

Passing  Rates: 

High  School  Male 

73.4 % 

High  School  Female 

88.0% 

90.6% 

93.3% 

95.5% 

College  Male 

86.3% 

89.3% 

92.4% 

94.8% 

College  Female 

94.4% 

95.7% 

97.0% 

98.0% 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
high  school  educated  Regular  Anny  Soldiers  by  gender.  For  a  particular  ST  score,  trace 
a  vertical  line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  detennine  the 
corresponding  probability  of  passing.  For  example,  a  ST  score  of  100  corresponds  to  a 
passing  probability  of  about  43%  for  a  high  school  educated,  Regular  Army  male  Soldier. 
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Chance  of  Passing 


Figure  1:  Predicted  Probability  of  Passing:  High  School  Soldiers 


^^^High  School  Male  High  School  Female 


Figure  2  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
male  Soldiers  by  education  level.  The  graph  illustrates  that  college  educated  Soldiers 
have  a  noticeably  better  chance  of  passing  this  course. 
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Figure  2:  Predicted  Probability  of  Passing:  Male  Soldiers 


"  College  Male  ^^^High  School  Male 


Figure  3  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
Soldiers  by  education  and  gender.  The  graph  illustrates  that  women  with  some  college 
education  have  the  best  chance  at  passing  the  course,  while  high  school  educated  males, 
the  modal  students,  have  the  lowest  chance  of  passing.  Figure  3  also  illustrates  the 
magnitude  of  the  effect  brought  about  by  gender  and  education.  At  the  current  cutoff 
score  (ST  =  102)  college  educated  female  Soldiers  have  about  a  40  percentage  point 
better  chance  of  passing  the  course  compared  to  high  school  educated  males. 
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Appendix:  Soldier  Characteristics  (97B) 


Gender 

Outcome 

Male 

Female 

fail 

9.6% 

8.1% 

pass 

90.4% 

91.9% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

14.3% 

1 1 .9% 

4.6% 

pass 

85.7% 

88.1% 

95.4% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

9.3% 

7.6% 

pass 

90.7% 

92.4% 

— 
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97E:  Human  Intelligence  Collector 


The  final  sample  included  748  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (81.3%)  or  failed  for  academic  reasons  (18.7%)  were 
included  in  the  analysis  sample  of  748  Soldiers  because  we  assume  that  non-completion 
for  non-academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  of  the  Soldiers  were  male  (69%,  31%  female),  most  had  a 
high  school  diploma  but  not  more  education  (59%,  40%  some  college,  1%  GED  or  less), 
and  the  greatest  number  were  from  Regular  Army  (84%,  11%  National  Guard,  5%  Anny 
Reserve).  The  governing  AA  composite,  Skilled  Technical  (ST),  for  this  MOS  has  a 
cutoff  score  of  92;  the  sample  mean  is  121.01  (standard  deviation  =  10.275).  See  the 
Appendix  for  a  description  of  Soldier  characteristics  for  this  MOS  training  sample. 


Table  1.  97E:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

92.634*** 

Log  Likelihood 

628.568 

Nagelkerke  R  Square 

OO 

OO 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-7.721 

.000 

43.122*** 

GED 

.452 

1.572 

.300 

COLLEGE 

.618 

1.855 

7.402** 

GUARD 

1.521 

4.575 

12.610*** 

RESERVE 

.733 

2.081 

1.371 

GENDER 

.292 

1.339 

1.756 

ST 

.073 

1.076 

54.871*** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 


A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
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Table  1  and  indicate  that  a  model,  including  the  ST  composite,  as  well  as  education  level 
(GED  =  GED  or  less  education;  COLLEGE  =  some  college),  Army  component  (GUARD 
=  National  Guard,  RESERVE  =  Army  Reserve),  and  gender  (GENDER  =  female), 
accounts  for  about  19%  of  the  variation  in  the  dependent  variable.  This  model  is 
statistically  significant  (y2  =  92.634,  P  <  .001),  with  moderate  explanatory  power 
(Nagelkerke  R2  =  .188). 

There  is  a  statistically  significant  effect  for  the  ST  composite.  The  model  suggests 
that  having  a  higher  ST  composite  score  increases  the  odds  of  passing.  At  the  mean  ST 
score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  eight  percent  in  the 
odds  of  passing  the  course,  and  a  five-point  increase  in  ST  would  increase  the  odds  of 
passing  by  44%.  There  are  also  significant  effects  for  COLLEGE  where  those  with  some 
college  have  86%  increased  odds  of  passing,  and  Army  component  where  Guardsmen 
and  Reservists  have  increased  odds  of  passing  (356%  and  108%,  respectively). 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

qw'x)  /  (1  +  q(P’x>). 

This  analysis  is  confined  to  male  Soldiers  from  the  Regular  Army  with  a  high  school  or 
college  education  (though  results  would  be  similar  for  other  demographic  groups).  Based 
on  the  model,  when  the  cutoff  score  is  at  its  current  level  (ST  =  92),  regular  Army 
Soldiers  with  a  high  school  diploma  and  an  average  ST  score  (ST  =  109.34)  have 
approximately  a  56%  chance  of  passing;  college  students  have  a  71%  chance.  Currently, 
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about  89%  of  Soldiers  are  eligible  for  MOS  97E  assignment  at  the  current  cutoff  (ST  = 
92).  Lowering  the  cutoff  by  five  points  (ST  =  87)  would  increase  eligibility  by  ten 
percentage  points  (to  97%)  and  the  average  Soldier  with  a  high  school  diploma  who 
would  qualify  for  training  would  have  a  slightly  lower  chance  of  passing  (54%);  the  same 
is  true  for  those  with  some  college  (68%).  Raising  the  cutoff  score  by  five  points  (ST  = 
97),  the  average  Soldier  with  a  high  school  diploma  would  have  a  slightly  higher  chance 
of  passing  (60%),  as  would  the  average  Soldier  with  some  college  (74%),  but  eligibility 
would  fall  by  about  ten  percentage  points. 


Table  2.  97E:  Probabi 
population)  will  pass  t 

lity  that  a  Regular  Army  Soldier  (from  the  larger  Army  contract 
le  course  based  on  the  binary  logistic  model 

Cutoff  = 

82 

Cutoff = 

87 

Cutoff = 

92 

Cutoff = 

97 

Cutoff = 
102 

Percent  Eligible 
(Regular  Army) 

99.5% 

96.8% 

89.3% 

78.8% 

64.8%o 

Mean 

107.14 

107.78 

109.34 

111.36 

114.05 

Passing  rate: 

High  school  male 

52.5% 

53.7% 

56.5% 

60.1% 

64.7% 

College  male 

67.2% 

68.2% 

70.7% 

73.6% 

77.3% 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
Male  Regular  Anny  Soldiers,  including  upper  and  lower  bounds  based  upon  the  standard 
error  of  the  estimated  ST  coefficient.  For  a  particular  ST  score,  trace  a  vertical  line  up  to 
the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the  corresponding 
probability  of  passing.  For  example,  an  ST  score  of  100  corresponds  to  a  passing 
probability  of  about  50%  for  a  high  school  educated,  Regular  Army  male  Soldier.  One 
can  see  that  Soldiers  with  a  relatively  low  ST  score  have  a  low  chance  of  passing  the 
course.  As  ST  scores  increase,  the  chance  that  a  Soldier  will  pass  the  course  increases 
sharply. 
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Figure  1:  Predicted  Probalility  of  Passing:  Male  Soldiers 


Appendix:  Soldier  Characteristics  (97E) 


Gender 

Male 

Female 

fail 

18.4% 

19.5% 

pass 

81.6% 

80.5% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

18.2% 

23.5% 

1 1 .7% 

pass 

81.8% 

76.5% 

88.3% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

20.7% 

8.8% 

8.2% 

pass 

79.3% 

91 .2% 

91.8% 
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97E  Risk  Analysis  of  Policy  Alternatives 


In  this  section  we  extend  the  policy  analysis  with  a  risk  analysis  of  the  policy 
alternatives  considered  using  simulation  methods.  We  employ  the  “@RISK”  software 
package  which  is  designed  as  an  add-in  for  Microsoft  Excel.  There  are  four  steps  in  the 
risk  analysis. 

The  first  step  is  developing  a  model.  A  regression  model  for  predicting  the 
passing  rate  has  been  specified  and  estimated,  and  the  results  stored  in  Excel  worksheet 
format.  See  Table  1. 

The  second  step  is  identifying  the  uncertainty  inherent  in  both  the  input  variables 
(composite  scores,  Soldier  characteristics)  and  the  corresponding  estimated  parameters 
and  specifying  their  possible  values  with  probability  distributions. 

To  approximate  the  governing  composite  distribution,  we  use  the  larger  Army 
contract  population  (EAF  file:  those  who  contracted  from  Jan  1992  to  Aug  2003).  This 
distribution  can  be  assumed  to  follow  a  normal  distribution.  The  mean  and  standard 
deviation  estimated  from  the  contract  population  and  used  as  input  to  the  simulations  are 
shown  in  Table  R-l  (before  truncation). 


Table  R-l.  Input  infonnation  for  the  simulations 

Mean 

Std  error  / 
deviation 

Constant 

-7.721 

1.176 

ST  var 

107.02  (before  truncation) 

12.157 

ST  coeff 

.073 

.010 

GED  var 

1.5% 

GED  coeff 

.452 

.825 

COLLEGE  var 

39.8% 

COLLEGE  coeff 

.618 

.227 

GUARD  var 

11.4% 

GUARD  coeff 

1.521 

.428 

RESERVE  var 

4.5% 

RESERVE  coeff 

.733 

.626 

GENDER  var 

30.9% 

GENDER  coeff 

.292 

.220 

76  Software  is  available  from  Palisade  Corporation,  Newfield,  NY. 
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To  approximate  the  distributions  for  the  demographic  variables  we  rely  on  the 
training  data  sample  used  to  estimate  the  regression  model.  These  distributions  can  be 
assumed  to  follow  binomial  distributions  with  means  estimated  from  the  training  data 
sample  as  shown  in  Table  R-l. 

To  approximate  the  distributions  for  the  estimated  parameters  in  the  logit  model, 
we  note  that,  under  very  general  conditions,  the  estimates  will  be  asymptotically  normally 
distributed.77  The  estimated  coefficients  (means  and  standard  errors)  are  shown  in  Table 
R-l. 


The  third  step  is  analyzing  the  model  with  simulation  to  determine  the  range  and 
probabilities  of  all  possible  training  outcome  results.  The  results  reported  represent  the 

70 

simulation  of  the  regression  prediction  equation  for  1000  iterations  using  a  fixed  seed. 

The  comparisons  are  between  the  baseline  case  (ST  =  92)  and  the  policy  cases  (ST  =  87 
and  ST  =  97).  These  cases  are  delineated  by  the  cutoff  score  level  which  serves  as  the 
lower  truncation  point  in  the  governing  composite  input  distribution.  The  simulation 
results  in  Table  R-2  describe  the  effect  on  the  mean  probability  of  passing  as  the  ST 
cutoff  score  level  is  varied,  and  represent  an  improvement  of  the  policy  analysis 
presented  in  Table  2  because  they  reflect  the  simultaneous  effects  of  all  known  sources  of 
uncertainty. 


Table  R-2.  97E  simulation  resul 

Its 

ST  =  87 

ST  =  92 

C/3 

H 

II 

80 

-0 

Mean  governing 
composite 

108.32 

109.54 

111.35 

Mean  probability 
of  passing 

67.90% 

69.33% 

71.99% 

Std  deviation 

26.16% 

27.20% 

27.98% 

The  frequency  distributions  of  simulated  effects  are  shown  below  in  the  histogram 
graphs.  The  histogram  portrays  the  full  range  of  outcomes  that  average  to  the  mean 
effect  as  reported.  Looking  at  the  histogram  for  the  baseline  case  (ST  =  92),  probability 
of  passing  is  plotted  on  the  horizontal  axis  and  the  height  of  each  bar  indicates  the 
relative  number  of  Soldiers  at  that  passing  probability.  The  information  in  the  histograms 
is  effectively  summarized  in  Table  R-3.  In  that  table  we  see  the  percentage  of  Soldiers 


77  The  intuition  is  that  logit  estimates  are  derived  from  maximum  likelihood  (ML)  estimation,  and  under 
fairly  general  conditions  ML  estimates  are  consistent  and  asymptotically  normally  distributed. 

78  Latin  Hypercube  (LH)  sampling  is  utilized  in  drawing  samples  for  the  input  distributions.  It  is  designed 
to  accurately  recreate  the  input  distribution  through  sampling  in  fewer  iterations  when  compared  with  the 
Monte  Carlo  method.  The  key  to  LH  sampling  is  stratification  of  the  input  probability  distributions. 
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who  meet  or  exceed  a  given  chance  of  passing.  For  example,  looking  at  the  baseline  case 
(ST  =  92),  about  58%  of  Soldiers  have  a  70%  or  greater  chance  of  passing. 


Table  R-3. 

Percent  of  simulated  Soldiers  that  meet  or  exceed 
given  chance  of  passing 


Chance  of 
passing 

ST  =  87 

ST  =  92 

ST  =  97 

95.0% 

17.63% 

18.58% 

20.63% 

90.0% 

29.90% 

33.43% 

33.56% 

85.0% 

38.82% 

41.39% 

43.45% 

80.0% 

47.21% 

46.65% 

50.65% 

75.0% 

53.33% 

52.63% 

56.98% 

70.0% 

58.34% 

57.75% 

61.65% 

65.0% 

62.18% 

62.29% 

66.05% 

60.0% 

66.93% 

66.39% 

70.10% 

55.0% 

71.40% 

70.47% 

73.50% 

50.0% 

53.33% 

52.63% 

56.98% 

We  find  that  the  mean  probability  of  passing  in  the  simulation  is  69%  (Table  R-2, 
baseline).  Looking  at  the  histograms,  we  see  the  largest  portion  of  Soldiers  is  in  the 
highest  range  of  passing  scores,  trending  downward  as  the  probability  of  passing 
approaches  zero.  We  note  that  the  mean  simulated  passing  probability  is  higher  but 
comparable  to  the  static  prediction  at  the  mean  ST  score  (Table  2).  For  the  static 
prediction  we  reported  the  mean  probability  of  passing  at  56.5%  for  male  high  school  and 
at  70%  for  male  with  (some)  college.  The  simulation  prediction  is  strongly  boosted  by 
COLLEGE  and  GUARD  composition  and  effects  (and  to  a  lesser  extent  by  RESERVE 
and  GENDER  effects).  In  the  simulated  predictions  we  are  calculating  an  average  over 
all  Soldiers  and  taking  into  account  all  sources  of  uncertainty,  whereas  in  the  static 
prediction  we  are  calculating  the  probability  of  passing  only  at  the  mean  ST  score, 
ignoring  the  distribution  of  Soldier  scores  etc.  as  well  as  uncertainty.  While  recognizing 
this  variability  is  important,  we  also  note  that  the  policy  case  impacts  relative  to  the 
baseline  are  the  same  in  both  simulation  and  static  prediction  modes. 

The  fourth  step  is  making  a  decision  based  on  the  results  provided  and  the 
preferences  of  the  school  proponent.  This  involves  the  evaluation  by  the  school 
proponent  of  the  tradeoff  between  increased  eligibility  and  reduced  passing  probabilities 
(and  vice-versa). 
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Distribution  for  PASS  EQUATION  102 


0  0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9  1 


Distribution  for  PASS  EQUATION  107 


0  0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9  1 
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Distribution  for  PASS  EQUATION  97 


0  0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9  1 
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98C:  Signals  Intelligence  Analyst 


The  final  sample  included  1621  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (81.4%)  or  failed  for  academic  reasons  (18.6%)  were 
included  in  the  analysis  sample  of  1621  Soldiers  because  we  assume  that  non-completion 
for  non-academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (76%,  24%  female),  most  had  a  high  school 
diploma  but  not  more  education  (74%,  25%  some  college,  1%  GED  or  less),  and  almost 
all  were  Regular  Army  (97%,  1%  National  Guard,  2%  Reserve).  The  governing  AA 
composite,  Skilled  Technical  (ST),  for  this  MOS  has  a  cutoff  score  of  102;  the  sample 
mean  is  119.16  (standard  deviation  =  9.0).  See  the  Appendix  for  a  description  of  Soldier 
characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 


Table  1.  98C:  Results  of  the  binary  logistic 
prediction  model 

Chi-  Square 

199.353*** 

Log  Likelihood 

1359.466 

Nagelkerke  R  Square 

.187 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-11.818 

.000 

121.259 

GED 

-.910 

.402 

1.948 

COLLEGE 

.045 

1.047 

.069 

RESERVE 

-.928 

.395 

2.874 

GUARD 

-.479 

.620 

.546 

GENDER 

.292 

1.339 

3.277 

ST 

.113 

1.120 

149.195*** 

*  =  p  <  .05  ***  =  p<.00] 

**  =  p  <  .005 

[ 
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Table  1  and  indicate  that  the  model,  including  the  ST  composite,  education  level  (GED  = 
GED  or  less  than  high  school  diploma;  COLLEGE  =  some  college)  ,  Army  component 
(GUARD  =  National  Guard;  RESERVE  =  Army  Reserve)  and  gender  (GENDER  = 
female),  accounts  for  about  19%  of  the  variation  in  the  dependent  variable.  This  model  is 
statistically  significant  (%2  =  199.353,  P  <  .001),  and  has  moderate  explanatory  power 
(Nagelkerke  R2  =  .187). 

There  is  a  statistically  significant  effect  for  the  ST  composite.  The  model 
suggests  that  having  a  higher  ST  composite  score  increases  the  odds  of  passing.  At  the 
mean  ST  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  12%  in 
the  odds  of  passing  the  course,  and  a  five-point  increase  in  ST  would  increase  the  odds  of 
passing  by  76%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  formula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

Q(P’X>  /  (1  + 

Because  GED,  COLLEGE,  GUARD,  RESERVE,  and  GENDER  were  not  statistically 
significant,  this  analysis  is  confined  to  male  Soldiers  with  a  high  school  diploma  from  the 
Regular  Anny  (though  results  would  be  similar  for  other  demographic  groups).  Based  on 
the  model,  when  the  cutoff  score  is  at  its  current  level  (ST  =  102),  regular  Anny  Soldiers 
with  an  average  ST  score  (ST  =  1 14.16)  have  approximately  a  75%  chance  of  passing. 
Currently,  about  65%  of  Soldiers  are  eligible  for  MOS  98C  assignment  at  the  current 
cutoff  (ST  =  102).  Lowering  the  cutoff  by  five  points  (ST  =  97)  would  increase 
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eligibility  by  13  percentage  points  (to  78%)  while  the  average  Soldier  who  would  qualify 
for  training  would  have  a  lower  chance  of  passing  (69%).  Raising  the  cutoff  score  by 
five  points  (ST  =107),  the  average  Soldier  would  have  a  higher  chance  of  passing  (81%) 
but  eligibility  would  fall  by  about  15  percentage  points  (50%). 


Table  2.  98C:  Probability  that  a  Soldier  (from  the  larger  Anny  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model 

Cutoff  = 

92 

Cutoff = 

97 

Cutoff = 
102 

Cutoff = 
107 

Cutoff = 
112 

Percent  Eligible 
(Regular  Army) 

89.0% 

78.2% 

64.9%> 

49.6% 

34.7% 

Mean 

109.45 

111.59 

114.16 

117.18 

120.51 

Passing  rate: 

High  School,  Male, 
Regular  Army 

63.4% 

68.8% 

74.7% 

80.6% 

85.8% 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
Regular  Army  Soldiers,  including  upper  and  lower  bounds  based  upon  the  standard  error 
of  the  estimated  ST  coefficient.  For  a  particular  ST  score,  trace  a  vertical  line  up  to  the 
curve,  and  then  a  horizontal  line  over  to  the  axis  to  determine  the  corresponding 
probability  of  passing.  For  example,  an  ST  score  of  100  corresponds  to  a  passing 
probability  of  about  37%  for  a  high  school  educated,  Regular  Army  male  Soldier.  One 
can  see  that  Soldiers  with  a  low  ST  score  stand  very  little  chance  of  passing  the  course. 
As  ST  scores  increase,  the  chance  that  a  Soldier  will  pass  the  course  increases  sharply. 
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Chi 


Figure  1:  Predicted  Probalility  of  Passing:  High  School  Male  Regular  Army  Soldiers 


Appendix:  Soldier  Characteristics  (98C) 


Gender 

Male 

Female 

Outcome 

fail 

17.3% 

23.0% 

pass 

82.7% 

77.0% 

Education  Level 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

pass 

36.4% 

63.6% 

20.1% 

79.9% 

13.8% 

86.2% 

Component 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

pass 

18.4% 

81.6% 

30.0% 

70.0% 

22.2% 

77.8% 
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98C  Risk  Analysis  of  Policy  Alternatives 


In  this  section  we  extend  the  policy  analysis  with  a  risk  analysis  of  the  policy 
alternatives  considered  using  simulation  methods.  We  employ  the  “@RISK”  software 
package  which  is  designed  as  an  add-in  for  Microsoft  Excel.  There  are  four  steps  in  the 
risk  analysis. 

The  first  step  is  developing  a  model.  A  regression  model  for  predicting  the 
passing  rate  has  been  specified  and  estimated,  and  the  results  stored  in  Excel  worksheet 
format.  See  Table  1. 

The  second  step  is  identifying  the  uncertainty  inherent  in  both  the  input  variables 
(composite  scores,  Soldier  characteristics)  and  the  corresponding  estimated  parameters 
and  specifying  their  possible  values  with  probability  distributions. 

To  approximate  the  governing  composite  distribution,  we  use  the  larger  Army 
contract  population  (EAF  file:  those  who  contracted  from  Jan  1992  to  Aug  2003).  This 
distribution  can  be  assumed  to  follow  a  normal  distribution.  The  mean  and  standard 
deviation  estimated  from  the  contract  population  and  used  as  input  to  the  simulations  are 
shown  in  Table  R-l  (before  truncation). 


Table  R-l.  Input  infonnation  for  the  simulations 

Mean 

Std  error  / 
deviation 

Constant 

-11.818 

1.073 

ST  var 

107.02  (before  truncation) 

12.157 

ST  coeff 

.113 

.009 

GED  var 

0.6% 

GED  coeff 

-.910 

.652 

COLLEGE  var 

25.7% 

COLLEGE  coeff 

.045 

.173 

GUARD  var 

1.3% 

GUARD  coeff 

-.479 

.648 

RESERVE  var 

1.5% 

RESERVE  coeff 

-.928 

.548 

GENDER  var 

24.4% 

GENDER  coeff 

.292 

.161 

79  Software  is  available  from  Palisade  Corporation,  Newfield,  NY. 
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To  approximate  the  distributions  for  the  demographic  variables  we  rely  on  the 
training  data  sample  used  to  estimate  the  regression  model.  These  distributions  can  be 
assumed  to  follow  binomial  distributions  with  means  estimated  from  the  training  data 
sample  as  shown  in  Table  R-l. 

To  approximate  the  distributions  for  the  estimated  parameters  in  the  logit  model, 
we  note  that,  under  very  general  conditions,  the  estimates  will  be  asymptotically  normally 
distributed.80  The  estimated  coefficients  (means  and  standard  errors)  are  shown  in  Table 
R-l. 


The  third  step  is  analyzing  the  model  with  simulation  to  determine  the  range  and 
probabilities  of  all  possible  training  outcome  results.  The  results  reported  represent  the 

o  1 

simulation  of  the  regression  prediction  equation  for  1000  iterations  using  a  fixed  seed. 

The  comparisons  are  between  the  baseline  case  (ST  =  102)  and  the  policy  cases  (ST  =  97 
and  ST  =  107).  These  cases  are  delineated  by  the  cutoff  score  level  which  serves  as  the 
lower  truncation  point  in  the  governing  composite  input  distribution.  The  simulation 
results  in  Table  R-2  describe  the  effect  on  the  mean  probability  of  passing  as  the  ST 
cutoff  score  level  is  varied,  and  represent  an  improvement  of  the  policy  analysis 
presented  in  Table  2  because  they  reflect  the  simultaneous  effects  of  all  known  sources  of 
uncertainty. 


Table  R-2.  98C  simulation  resul 

Its 

C/5 

H 

II 

"'4 

ST  =  102 

ST  =  107 

Mean  governing 
composite 

111.36 

113.76 

116.71 

Mean  probability 
of  passing 

62.8% 

66.9% 

72.1% 

Std  deviation 

28.5% 

27.0% 

24.7% 

The  frequency  distributions  of  simulated  effects  are  shown  below  in  the  histogram 
graphs.  The  histogram  portrays  the  full  range  of  outcomes  that  average  to  the  mean 
effect  as  reported.  Looking  at  the  histogram  for  the  baseline  case  (ST  =  102),  probability 
of  passing  is  plotted  on  the  horizontal  axis  and  the  height  of  each  bar  indicates  the 
relative  number  of  Soldiers  at  that  passing  probability.  The  information  in  the  histograms 
is  effectively  summarized  in  Table  R-3.  In  that  table  we  see  the  percentage  of  Soldiers 


SH  The  intuition  is  that  logit  estimates  are  derived  from  maximum  likelihood  (ML)  estimation,  and  under 
fairly  general  conditions  ML  estimates  are  consistent  and  asymptotically  normally  distributed. 
sl  Latin  Hypercube  (LH)  sampling  is  utilized  in  drawing  samples  for  the  input  distributions.  It  is  designed 
to  accurately  recreate  the  input  distribution  through  sampling  in  fewer  iterations  when  compared  with  the 
Monte  Carlo  method.  The  key  to  LH  sampling  is  stratification  of  the  input  probability  distributions. 
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who  meet  or  exceed  a  given  chance  of  passing.  For  example,  looking  at  the  baseline  case 
(ST  =  102),  about  54%  of  Soldiers  have  a  70%  or  greater  chance  of  passing. 


Table  R-3. 

Percent  of  simulated  Soldiers  that  meet  or  exceed 
given  chance  of  passing 


Chance  of 
passing 

ST  =  97 

ST  =  102 

ST  =  107 

95.0% 

11.8% 

15.5% 

19.3% 

90.0% 

23.0% 

26.6% 

30.6% 

85.0% 

29.8% 

35.7% 

40.8% 

80.0% 

36.6% 

42.4% 

48.9% 

75.0% 

42.9% 

49.0% 

56.3% 

70.0% 

48.9% 

53.5% 

63.2% 

65.0% 

54.8% 

57.3% 

68.0% 

60.0% 

58.6% 

62.6% 

73.1% 

55.0% 

62.7% 

68.2% 

77.5% 

50.0% 

65.7% 

73.2% 

81.0% 

We  find  that  the  mean  probability  of  passing  in  the  simulation  is  67%  (Table  R-2, 
baseline).  Looking  at  the  histograms,  we  see  the  largest  portion  of  Soldiers  is  in  the 
highest  range  of  passing  scores,  trending  downward  as  the  probability  of  passing 
approaches  zero.  We  note  that  the  mean  simulated  passing  probability  is  lower  but 
comparable  to  the  static  prediction  at  the  mean  ST  score  (Table  2:  75%  for  male  high 
school  Soldiers).  In  the  simulated  predictions  we  are  calculating  an  average  over  all 
Soldiers  and  taking  into  account  all  sources  of  uncertainty,  whereas  in  the  static 
prediction  we  are  calculating  the  probability  of  passing  only  at  the  mean  ST  score, 
ignoring  the  distribution  of  Soldier  scores  etc.  as  well  as  uncertainty.82  While 
recognizing  this  variability  is  important,  we  also  note  that  the  policy  case  impacts  relative 
to  the  baseline  are  the  same  in  both  simulation  and  static  prediction  modes. 

The  fourth  step  is  making  a  decision  based  on  the  results  provided  and  the 
preferences  of  the  school  proponent.  This  involves  the  evaluation  by  the  school 
proponent  of  the  tradeoff  between  increased  eligibility  and  reduced  passing  probabilities 
(and  vice-versa). 


S2  For  several  other  MOS  sensitivity  analysis  indicates  that  we  can  attribute  this  difference  (between 
simulation  and  static  predictions)  to  the  relatively  large  standard  errors  on  the  constant  and  governing 
composite  terms  that  come  into  play  in  the  simulation.  In  the  present  case,  however,  the  standard  errors  are 
not  especially  large  and  so  the  causes  of  the  difference  are  not  apparent. 
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Distribution  for  PASS  EQUATION  102 


X  <=0.15  X  <=0.98 


Distribution  for  PASS  EQUATION  107 


X<=0.19  X  <=0.99 
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Distribution  for  PASS  EQUATION  97 


X  <=0.12  X  <=0.98 
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98G:  Voice  Interceptor 


The  final  sample  included  1461  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (92.3%)  or  failed  for  academic  reasons  (7.7%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (67%,  33%  female),  most  had  a  high  school 
diploma  but  not  more  education  (65%,  34%  some  college,  1%  GED  or  less),  and  greatest 
number  were  from  Regular  Army  (96%,  4%  National  Guard,  <  1%  Reserve).  The 
governing  AA  composite,  Skilled  Technical  (ST),  for  this  MOS  has  a  cutoff  score  of  92; 
the  sample  mean  is  122.21  (standard  deviation  =  9.697).  See  the  Appendix  for  a 
description  of  Soldier  characteristics  for  this  MOS  training  sample. 

A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 


Table  1.  98G:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

5.385 

Log  Likelihood 

790.085 

Nagelkerke  R  Square 

.009 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

2.030 

7.614 

2.360 

GED 

-1.520 

.219 

3.196 

COLLEGE 

-.306 

.737 

2.260 

GUARD 

-.040 

.961 

.006 

GENDER 

-1.69 

.844 

.603 

ST 

.005 

1.005 

.241 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p<  .005 
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Table  1  and  indicate  that  a  model,  including  the  ST,  education  level  (GED  =  GED  or  less 
than  high  school  diploma;  COLLEGE  =  some  college)  and  Army  component  (GUARD  = 
National  Guard),  accounts  for  almost  one  percent  of  the  variation  in  the  dependent 
variable.  This  model  is  not  statistically  significant  (x  =  53.85,  P  <  .001),  and  has  little 
explanatory  power  (Nagelkerke  R2  =  .009). 

Policy  Analysis.  Because  the  model  estimated  for  98G  is  not  statistically 
significant  and  has  little  explanatory  power,  no  policy  analysis  can  be  developed  from  the 
existing  data. 
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Appendix:  Soldier  Characteristics  (98G) 


Gender 

Outcome 

Male 

Female  | 

fail 

8.5% 

pass 

91.5% 

Education  Level  j 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

25.0% 

6.9% 

pass 

75.0% 

93.1% 

91.0% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

7.7% 

8.8% 

92.3% 

91.2% 
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98H:  Communications  Interceptor/Locator 


The  final  sample  included  474  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (41.7%)  or  failed  for  academic  reasons  (58.6%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (58%,  43%  female),  most  had  a  high  school 
diploma  but  not  more  education  (89%,  10%  some  college,  1%  GED  or  less),  and  virtually 
all  were  from  the  Regular  Anny  (99%,  1%  National  Guard).  The  governing  AA 
composite,  Skilled  Technical  (ST),  for  this  MOS  has  a  cutoff  score  of  92;  the  sample 
mean  is  104.55  (standard  deviation  =  1 1.01 1).  See  the  Appendix  for  a  description  of 
Soldier  characteristics  for  this  MOS  training  sample. 


Table  1.  98H:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

17.968*** 

Log  Likelihood 

626.242 

Nagelkerke  R  Square 

.0 

150 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-4.532 

.011 

16.188*** 

COLLEGE 

.042 

1.043 

.016 

GENDER 

-.001 

.999 

.000 

ST 

.040 

1.041 

14.642*** 

*  =  p  <  .05  ***  =  p  <  .001 


A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  model,  including  the  ST  composite,  education  status 
(COLLEGE)  and  gender  (GENDER  =  female),  accounts  for  about  5%  of  the  variation  in 
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2 

the  dependent  variable.  This  model  is  statistically  significant  (y  =  17.968,  P  <  .001),  but 
has  limited  explanatory  power  (Nagelkerke  R"  =  .050). 

There  are  statistically  significant  effects  for  the  ST  composite.  The  model 
suggests  that  having  a  higher  ST  composite  score  increases  the  odds  of  passing.  At  the 
mean  ST  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  4  percent 
in  the  odds  of  passing  the  course,  and  a  five-point  increase  in  ST  would  increase  the  odds 
of  passing  by  22%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  fonnula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

Q(P’x>  /  (1  +  Q(P’x)y 

Because  COLLEGE  and  GENDER  were  not  statistically  significant,  this  analysis  is 
confined  to  the  modal  demographic — high  school  educated  male  Soldiers  (though  results 
would  be  similar  for  any  demographic  combination).  Based  on  the  model,  when  the 
cutoff  score  is  at  its  current  level  (ST  =  92),  male  Soldiers  with  a  high  school  diploma 
and  an  average  ST  score  (ST  =  109.34)  have  approximately  a  46%  chance  of  passing. 
Currently,  about  89%  of  Soldiers  are  eligible  for  MOS  98H  assignment  at  the  current 
cutoff  (ST  =  92).  Lowering  the  cutoff  by  five  points  (ST  =  87)  would  increase  eligibility 
by  eight  percentage  points,  while  the  average  male  Soldier  who  would  qualify  for  training 
would  have  a  slightly  lower  chance  of  passing  (male  =  45%).  Raising  the  cutoff  score  by 
five  points  (ST  =  97),  the  average  male  Soldier  who  would  still  qualify  for  the  MOS 
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Table  2.  98H:  Probability  that  a  Soldier  (from  the  larger  Army  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model 

Cutoff  = 

82 

Cutoff = 

87 

Cutoff = 

92 

Cutoff = 

97 

Cutoff = 
102 

Percent  Eligible 
(Regular  Army) 

99.5% 

96.8% 

89.3% 

78.8% 

64.8%o 

Mean 

107.14 

107.78 

109.34 

111.36 

114.05 

Passing  rates: 

Male  High  School 

43.9% 

44.5% 

46.0% 

48.1% 

50.7% 

would  have  a  slightly  higher  chance  of  passing  (48%),  but  fewer  Soldiers  would  be 
eligible  (79%). 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
male  Soldiers,  including  upper  and  lower  bounds  based  upon  the  standard  error  of  the 
estimated  ST  coefficient.  For  a  particular  ST  score,  trace  a  vertical  line  up  to  the  curve, 
and  then  a  horizontal  line  over  to  the  axis  to  determine  the  corresponding  probability  of 
passing.  For  example,  an  ST  value  of  100  corresponds  to  a  passing  probability  of  about 
37%  for  a  male  Soldier.  One  can  see  that  the  probability  of  passing  increases  steadily 
through  the  range  of  the  data. 
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Chance  of  Passing 


Figure  1 :  Predicted  Probalility  of  Passing:  Male  High  School  Soldiers 


■  Male  -  -  -  Male  Upper  -  "  ■  Male  Lower 
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Appendix:  Soldier  Characteristics  (98H) 


Gender 

Outcome 

Male 

Female 

fail 

55.7% 

61.9% 

pass 

44.3% 

38.1% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

59.4% 

51.1% 

pass 

80.0% 

40.6% 

48.9% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

58.2% 

.0% 

75.0% 

pass 

41.8% 

.0% 

25.0% 
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98 J:  Electronic  Intelligence  Analyst 


The  final  sample  included  503  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (88.7%)  or  failed  for  academic  reasons  (1 1.3%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (70%,  30%  female),  most  had  a  high  school 
diploma  but  not  more  education  (86%,  12%  some  college,  2%  GED  or  less),  and  most 
were  from  the  Regular  Army  (93%,  1%  Guard,  6%  Reserve).  The  governing  AA 
composite,  Skilled  Technical  (ST),  for  this  MOS  has  a  cutoff  score  of  102;  the  sample 
mean  is  1 15.36  (standard  deviation  =  8.708).  See  the  Appendix  for  a  description  of 
Soldier  characteristics  for  this  MOS  training  sample. 


Table  1.  98J:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

23.41 1*** 

Log  Likelihood 

332.081 

Nagelkerke  R  Square 

.090 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-7.222 

.001 

9.461** 

GED 

-.359 

.698 

.153 

COLLEGE 

1.278 

3.590 

2.971 

RESERVE 

-.106 

.899 

.026 

GENDER 

.241 

1.273 

.567 

ST 

.081 

1.084 

15.125*** 

*  =  p  <  .05  ***=p<.001 

**  =p  <  .01 


A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
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Table  1  and  indicate  that  the  model,  including  the  ST  composite,  education  level  (GED  = 
GED  or  less  education,  COLLGE  =  some  college),  Army  component  (RESERVE  = 

Army  Reserve),  and  gender  (GENDER  =  female),  accounts  for  about  9%  of  the  variation 
in  the  dependent  variable.  This  model  is  statistically  significant  (x  =  23.41 1,  P  <  .001), 
and  has  moderate  explanatory  power  (Nagelkerke  R2  =  .090). 

There  are  statistically  significant  effects  for  the  ST  composite.  The  model 
suggests  that  having  a  higher  ST  composite  score  increases  the  odds  of  passing.  At  the 
mean  ST  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  eight 
percent  in  the  odds  of  passing  the  course,  and  a  five-point  increase  in  ST  would  increase 
the  odds  of  passing  by  50%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  fonnula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

qw'x)  /  (1  +  q(P’x>). 

Because  GED,  COLLEGE,  and  GENDER  were  not  statistically  significant,  this  analysis 
is  confined  to  the  modal  demographic — high  school  educated  male  Soldiers  (though 
results  would  be  similar  for  any  demographic  combination).  Based  on  the  model,  when 
the  cutoff  score  is  at  its  current  level  (ST  =  102),  male  Soldiers  with  a  high  school 
diploma  and  an  average  ST  score  (ST  =  1 14.05)  have  approximately  an  88%  chance  of 
passing.  Currently,  about  65%  of  Soldiers  are  eligible  for  MOS  98J  assignment  at  the 
current  cutoff  (ST  =  102).  Lowering  the  cutoff  by  five  points  (ST  =  97)  would  increase 
eligibility  by  six  percentage  points,  while  the  average  male  Soldier  who  would  qualify  for 
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training  would  have  a  slightly  lower  chance  of  passing  (male  =  86%).  Raising  the  cutoff 
score  by  five  points  (ST  =  107),  the  average  male  Soldier  who  would  still  qualify  for  the 
MOS  would  have  a  slightly  higher  chance  of  passing  (91%),  but  fewer  Soldiers  would  be 
eligible  (50%). 


Table  2.  98 J:  Probability  that  a  Soldier  (from  the  larger  Anny  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model 

Cutoff  = 

92 

Cutoff = 

97 

Cutoff = 
102 

Cutoff = 
107 

Cutoff = 
112 

Percent  Eligible 
(Regular  Army) 

89.3% 

78.8% 

64.8%o 

49.6 

36.2 

Mean 

Passing  rates: 

109.34 

111.36 

114.05 

117.08 

High  school  male 

83.7% 

85.6% 

88.2% 

90.6% 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
high  school  educated  male  Soldiers,  including  upper  and  lower  bounds  based  upon  the 
standard  error  of  the  estimated  ST  coefficient.  For  a  particular  ST  score,  trace  a  vertical 
line  up  to  the  curve,  and  then  a  horizontal  line  over  to  the  axis  to  detennine  the 
corresponding  probability  of  passing.  For  example,  an  ST  value  of  100  corresponds  to  a 
passing  probability  of  about  70%  for  a  high  school  educated  male  Soldier.  One  can  see 
that  the  probability  of  passing  increases  through  the  data  range. 
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Chance  of  Passing 


Figure  1 :  Predicted  Probalility  of  Passing:  Male  High  School  Soldiers 


—  Male  -  -  -  Male  Upper  -  -  -  Male  Lower 
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Appendix:  Soldier  Characteristics  (98J) 


Gender 

Outcome 

Male 

Female 

fail 

10.8% 

12.7% 

pass 

89.2% 

87.3% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

12.2% 

3.4% 

pass 

80.0% 

87.8% 

96.6% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

11.3% 

13.8% 

.0% 

pass 

88.7% 

86.2% 

100.0% 
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98K:  Signal  Collector  &  Identifier 


The  final  sample  included  374  Soldiers  who  were  coded  as  either  graduates  or 
academic  failures  during  their  first  AIT  attempt  in  this  MOS  during  a  period  from  2001  to 
2004.  Only  Soldiers  who  graduated  (90.4%)  or  failed  for  academic  reasons  (9.6%)  were 
included  in  the  analysis  sample  because  we  assume  that  non-completion  for  non- 
academic  reasons  is  not  related  to  cognitive  criteria. 

In  the  sample,  most  were  male  (59%,  41%  female),  most  had  a  high  school 
diploma  but  not  more  education  (86%,  13%  some  college,  1%  GED  or  less),  and  all  were 
from  Regular  Army.  The  governing  AA  composite,  Skilled  Technical  (ST),  for  this  MOS 
has  a  cutoff  score  of  92;  the  sample  mean  is  104.17  (standard  deviation  =  9.618).  See  the 
Appendix  for  a  description  of  Soldier  characteristics  for  this  MOS  training  sample. 


Table  1.  98K:  Results  of  the  binary  logistic 
prediction  model _ _ 


Chi-  Square 

18.685*** 

Log  Likelihood 

218.266 

Nagelkerke  R  Square 

.104 

Variable 

Coefficient 

Odds 

Ratio 

Wald 

Constant 

-9.570 

.000 

8.517*** 

GENDER 

.236 

1.266 

.412 

ST 

.116 

1.123 

12.507** 

*  =  p  <  .05  ***  =  p  <  .001 

**  =  p  <  .01 


A  binary  logistic  regression  model  was  estimated  to  explain  pass/fail  AIT 
outcomes  and  focused  on  the  first  training  attempt.  The  estimation  results  are  reported  in 
Table  1  and  indicate  that  the  model,  including  the  ST  composite  and  gender  (GENDER  = 
female),  accounts  for  about  10%  of  the  variation  in  the  dependent  variable.  This  model  is 
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statistically  significant  (y2  =  18.685,  P  <  .001),  and  has  moderate  explanatory  power 
(Nagelkerke  R2  =  .104). 

There  are  statistically  significant  effects  for  the  ST  composite.  The  model 
suggests  that  having  a  higher  ST  composite  score  increases  the  odds  of  passing.  At  the 
mean  ST  score,  an  increase  of  one  point  is  associated  with  an  increase  of  about  twelve 
percent  in  the  odds  of  passing  the  course,  and  a  five-point  increase  in  ST  would  increase 
the  odds  of  passing  by  79%. 

Policy  Analysis.  Table  2  reports  the  probability  that  the  average  Soldier  from  the 
larger  Anny  contract  population  would  pass  the  course  based  upon  the  binary  logistic 
model.  These  were  calculated  using  the  fonnula  for  finding  probability  in  a  binary 
logistic  model,  which  is 

Q(P’x)  /  (1  +  Q(P’x)y 

Because  GENDER  was  not  statistically  significant,  this  analysis  is  confined  to  the  modal 
demographic — male  Soldiers  (though  results  would  be  similar  for  any  demographic 
combination).  Based  on  the  model,  when  the  cutoff  score  is  at  its  current  level  (ST  =  92), 
male  Soldiers  with  a  high  school  diploma  and  an  average  ST  score  (ST  =  109.34)  have 
approximately  a  96%  chance  of  passing.  Currently,  about  89%  of  Soldiers  are  eligible  for 
MOS  98K  assignment  at  the  current  cutoff.  Lowering  the  cutoff  by  five  points  (ST  =  87) 
would  increase  eligibility  by  eight  percentage  points,  while  the  average  Soldier  who 


Table  2.  98K:  Probability  that  a  Soldier  (from  the  larger  Army  contract  population) 
will  pass  the  course  based  on  the  binary  logistic  model _ _ _ 


Cutoff  = 

Cutoff = 

Cutoff = 

Cutoff = 

Cutoff = 

82 

87 

92 

97 

102 

Percent  Eligible 
(Regular  Army) 

99.5% 

96.8% 

89.3% 

78.8% 

64.8%o 

Mean 

Passing  rates: 

107.14 

107.78 

109.34 

111.36 

114.05 

High  school  male 

94.6% 

95.0% 

95.7% 

96.6% 

97.5% 
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would  qualify  for  training  would  have  about  the  same  chance  of  passing  (male  =  95%). 
Raising  the  cutoff  score  by  five  points  (ST  =  97),  the  average  Soldier  who  would  still 
qualify  for  the  MOS  would  have  about  the  same  chance  of  passing  (97%),  but  fewer 
Soldiers  would  be  eligible  (79%). 

Figure  1  shows  the  relationship  between  ST  and  the  probability  of  passing  for 
male  Soldiers,  including  upper  and  lower  bounds  based  upon  the  standard  error  of  the 
estimated  ST  coefficient.  For  a  particular  ST  score,  trace  a  vertical  line  up  to  the  curve, 
and  then  a  horizontal  line  over  to  the  axis  to  determine  the  corresponding  probability  of 
passing.  For  example,  an  ST  value  of  100  corresponds  to  a  passing  probability  of  about 
88%  for  a  male  Soldier.  One  can  see  that  the  probability  of  passing  increases  sharply 
through  the  bottom  of  the  data  range. 


Figure  1:  Predicted  Probalility  of  Passing:  Male  Soldiers 


-  -  -  Male  Upper  -  -  -  Male  Lower 
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Appendix:  Soldier  Characteristics  (98K) 


Gender 

Outcome 

Male 

Female 

fail 

9.1% 

10.4% 

pass 

90.9% 

89.6% 

Education  Level 

Outcome 

GED  or  less 

High  School 
Diploma 

Some  College 
or  More 

fail 

.0% 

10.6% 

4.0% 

pass 

100.0% 

89.4% 

96.0% 

Component 

Outcome 

Regular  Army 

Army 

Reserve 

National 

Guard 

fail 

9.6% 

.0% 

.0% 

pass 

90.4% 

.0% 

.0% 
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